Instrumenting nodejs with AddressSanitizer fails resulting of memory leaks #32835

zyscoder · 2020-04-14T02:10:48Z

Version: v12.16.0
Platform: Linux vul337 4.15.0-91-generic new design of error handling #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Subsystem:

What steps will reproduce the bug?

When building nodejs with AddressSanitizer,

CC="/path/to/afl-clang" CXX="/path/to/afl-clang++" LINK="/path/to/afl-clang++ -fuse-ld=lld" CCFLAGS="-fsanitize=address" CXXFLAGS="-fsanitize=address" LDFLAGS="-fsanitize=address" make

the following command will be run, and memory leaks occur:

LD_LIBRARY_PATH=/path/to/node/out/Release/lib.host:/path/to/node/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../.; mkdir -p /path/to/node/out/Release/obj/gen; "/path/to/node/out/Release/mkcodecache" "/path/to/node/out/Release/obj/gen/node_code_cache.cc"

How often does it reproduce? Is there a required condition?

No. This potential bug can always be reproduced.

What is the expected behavior?

The executable of 'mkcodecache' tries to generate the file of 'node_code_cache.cc' when building nodejs, and the building process fails due to memory leaks of 'mkcodecache'. This problem should be handled otherwise a nodejs instrumented by Address Sanitizer cannot be built successfully.

What do you see instead?

Multiple stack dumps of memory leaks:

==72826==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 16384 byte(s) in 4 object(s) allocated from:
    #0 0x3108ce1 in __interceptor_calloc (/path/to/node/out/Release/mkcodecache+0x3108ce1)
    #1 0x4821af9 in v8::internal::BasicMemoryChunk::BasicMemoryChunk(unsigned long, unsigned long, unsigned long) (/path/to/node/out/Release/mkcodecache+0x4821af9)
    #2 0x47f383b in v8::internal::MemoryChunk::Initialize(v8::internal::Heap*, unsigned long, unsigned long, unsigned long, unsigned long, v8::internal::Executability, v8::internal::Space*, v8::internal::VirtualMemory) (/path/to/node/out/Release/mkcodecache+0x47f383b)
    #3 0x47eb8ea in v8::internal::MemoryAllocator::AllocateChunk(unsigned long, unsigned long, v8::internal::Executability, v8::internal::Space*) (/path/to/node/out/Release/mkcodecache+0x47eb8ea)
    #4 0x47eca8d in v8::internal::Page* v8::internal::MemoryAllocator::AllocatePage<(v8::internal::MemoryAllocator::AllocationMode)1, v8::internal::SemiSpace>(unsigned long, v8::internal::SemiSpace*, v8::internal::Executability) (/path/to/node/out/Release/mkcodecache+0x47eca8d)
    #5 0x4801895 in v8::internal::SemiSpace::Commit() (/path/to/node/out/Release/mkcodecache+0x4801895)
    #6 0x480153c in v8::internal::NewSpace::NewSpace(v8::internal::Heap*, v8::PageAllocator*, unsigned long, unsigned long) (/path/to/node/out/Release/mkcodecache+0x480153c)
    #7 0x4687500 in v8::internal::Heap::SetUpSpaces() (/path/to/node/out/Release/mkcodecache+0x4687500)
    #8 0x456f8ca in v8::internal::Isolate::Init(v8::internal::ReadOnlyDeserializer*, v8::internal::StartupDeserializer*) (/path/to/node/out/Release/mkcodecache+0x456f8ca)
    #9 0x4571838 in v8::internal::Isolate::InitWithSnapshot(v8::internal::ReadOnlyDeserializer*, v8::internal::StartupDeserializer*) (/path/to/node/out/Release/mkcodecache+0x4571838)
    #10 0x543af7e in v8::internal::Snapshot::Initialize(v8::internal::Isolate*) (/path/to/node/out/Release/mkcodecache+0x543af7e)
    #11 0x41df7cb in v8::Isolate::Initialize(v8::Isolate*, v8::Isolate::CreateParams const&) (/path/to/node/out/Release/mkcodecache+0x41df7cb)
    #12 0x41dff9e in v8::Isolate::New(v8::Isolate::CreateParams const&) (/path/to/node/out/Release/mkcodecache+0x41dff9e)
    #13 0x347f816 in main (/path/to/node/out/Release/mkcodecache+0x347f816)
    #14 0x7f39921d8b96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
...

SUMMARY: AddressSanitizer: 118344 byte(s) leaked in 1206 allocation(s).
node.target.mk:13: recipe for target '/path/to/node/out/Release/obj/gen/node_code_cache.cc' failed
make[1]: *** [/path/to/node/out/Release/obj/gen/node_code_cache.cc] Error 1
make[1]: *** Waiting for unfinished jobs....
rm c37f73ea17746a11dc6852c9156cd99eb8565d70.intermediate 16d0a06d018bbaab79792789d3c3aadd33ec98cf.intermediate 3fe54b56864e2ab8dadc5290dbe04ab6d00cf977.intermediate 0459621a7970293331ab29d45be116d6b911f492.intermediate
Makefile:101: recipe for target 'node' failed
make: *** [node] Error 2

Additional information

The text was updated successfully, but these errors were encountered:

addaleax · 2020-04-14T02:23:43Z

--- a/tools/code_cache/mkcodecache.cc
+++ b/tools/code_cache/mkcodecache.cc
@@ -49,8 +49,8 @@ int main(int argc, char* argv[]) {
 
   // Create a new Isolate and make it the current one.
   Isolate::CreateParams create_params;
-  create_params.array_buffer_allocator =
-      ArrayBuffer::Allocator::NewDefaultAllocator();
+  create_params.array_buffer_allocator_shared.reset(
+      ArrayBuffer::Allocator::NewDefaultAllocator());
   Isolate* isolate = Isolate::New(create_params);
   {
     Isolate::Scope isolate_scope(isolate);
@@ -65,6 +65,7 @@ int main(int argc, char* argv[]) {
     out << cache;
     out.close();
   }
+  isolate->Dispose();
 
   v8::V8::ShutdownPlatform();
   return 0;

Does this help? If yes, feel free to open a PR with it.

zyscoder · 2020-04-14T02:30:33Z

Thank you for your timely reply. Let me have a try.

zyscoder · 2020-04-14T03:52:01Z

@addaleax Your patch works well!

mmarchini · 2020-04-14T04:41:08Z

The executable of 'mkcodecache' tries to generate the file of 'node_code_cache.cc' when building nodejs, and the building process fails due to memory leaks of 'mkcodecache'. This problem should be handled otherwise a nodejs instrumented by Address Sanitizer cannot be built successfully.

Interesting. We recently introduced ASAN checks on our CI (https://github.com/nodejs/node/actions?query=workflow%3Atest-asan), and so far it hasn't failed. It's worth noting we use ./configure --enable-asan && make instead of setting the flags via CC and family. I wonder what's different and if we need to change anything in our config to increase coverage (or if this issue is exclusive of v12). I don't think we disabled halt_on_error during the build phase, so mkcodecache should fail if there's a ASAN violation.

zyscoder · 2020-04-14T06:06:32Z

@addaleax The patch does work well on the master branch. But I found that in v12.16.0, 'CreateParams' has no member of 'array_buffer_allocator_shared'.

zyscoder · 2020-04-14T07:08:17Z

@mmarchini I tried to build v12.16.0 in your way and succeeded.

mmarchini · 2020-04-14T16:57:04Z

Ok, so something is different when building with the flag and setting things up manually, or maybe there's a compiler's difference. Either way, this looks like something our CI should've flagged, nice catch!

Fixes: nodejs#32835

addaleax · 2020-04-14T17:04:14Z

@zyscoder Right, on older release lines you may have to manually keep the reference to the ArrayBuffer::Allocator alive, with a separate std::unique_ptr or so. But as far as I am concerned, fixing this in master is good enough, as it’s not a “real” issue in the sense that Node.js users are affected by it.

Anyway, I’ve opened #32850 with the patch from above.

mmarchini · 2020-04-14T17:13:27Z

@zyscoder which clang version did you use?

zyscoder · 2020-04-15T15:07:14Z

@zyscoder which clang version did you use?

clang-8.0.0

Fixes: #32835 PR-URL: #32850 Reviewed-By: Matheus Marchini <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Juan José Arboleda <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

Fixes: nodejs#32835 PR-URL: nodejs#32850 Reviewed-By: Matheus Marchini <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Juan José Arboleda <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

Fixes: #32835 PR-URL: #32850 Backport-PR-URL: #33128 Reviewed-By: Matheus Marchini <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Juan José Arboleda <[email protected]> Reviewed-By: Jiawen Geng <[email protected]>

vdata1 · 2022-06-17T15:12:26Z

Hi @mmarchini , @addaleax

I tried building node JS with ASAN on Linux. I used ./configure --enable-asan && make and still ended up with crashes.
I used several versions of Ubuntu and Debian as an environment to build Node with ASAN.

Please find the error message below:

In file included from ../deps/v8/src/base/virtual-address-space.h:11,
                from ../deps/v8/src/base/emulated-virtual-address-subspace.h:13,
                from ../deps/v8/src/base/emulated-virtual-address-subspace.cc:5:
../deps/v8/src/base/platform/platform.h: In static member function 'static v8::base::Stack::StackSlot v8::base::Stack::GetRealStackAddressForSlot(v8::base::Stack::StackSlot)':
../deps/v8/src/base/platform/platform.h:635:16: error: operands to ?: have different types 'char*' and 'v8::base::Stack::StackSlot'
 634 |     return real_frame
     |            ~~~~~~~~~~
 635 |                ? (static_cast<char*>(real_frame) + kAsanRealFrameOffsetBytes)
     |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 636 |                : slot;
     |                ~~~~~~
../deps/v8/src/base/platform/platform.h:636:18: error: conversion from 'v8::base::Stack::StackSlot' to 'char*' is ambiguous
 636 |                : slot;
     |                  ^~~~
../deps/v8/src/base/platform/platform.h:608:5: note: candidate: 'v8::base::Stack::StackSlot::operator void*() const' <near match>
 608 |     operator void*() const { return reinterpret_cast<void*>(value); }
     |     ^~~~~~~~
../deps/v8/src/base/platform/platform.h:608:5: note:   no known conversion from 'void*' to 'char*'
../deps/v8/src/base/platform/platform.h:609:5: note: candidate: 'v8::base::Stack::StackSlot::operator uintptr_t() const' <near match>
 609 |     operator uintptr_t() const { return value; }  // NOLINT
     |     ^~~~~~~~
../deps/v8/src/base/platform/platform.h:609:5: note:   no known conversion from 'uintptr_t' {aka 'long unsigned int'} to 'char*'
make[1]: *** [tools/v8_gypfiles/v8_libbase.target.mk:194: /home/ubuntu/node/out/Release/obj.target/v8_libbase/deps/v8/src/base/emulated-virtual-address-subspace.o] Error 1
rm 1d4311d5329fc69c1bf42791f159aee2725574ef.intermediate
make: *** [Makefile:113: node] Error 2

F3n67u · 2022-06-17T15:20:50Z

@vdata1 see #43370

zyscoder changed the title ~~Instrumenting nodejs with AddressSanitizer failed results of memory leaks~~ Instrumenting nodejs with AddressSanitizer fails resulting of memory leaks Apr 14, 2020

addaleax added a commit to addaleax/node that referenced this issue Apr 14, 2020

tools: fix mkcodecache when run with ASAN

d9b5672

Fixes: nodejs#32835

addaleax mentioned this issue Apr 14, 2020

tools: fix mkcodecache when run with ASAN #32850

Closed

2 tasks

gengjiawen closed this as completed in 7be5f58 Apr 22, 2020

addaleax mentioned this issue Apr 28, 2020

[v12.x] tools: fix mkcodecache when run with ASAN #33128

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Instrumenting nodejs with AddressSanitizer fails resulting of memory leaks #32835

Instrumenting nodejs with AddressSanitizer fails resulting of memory leaks #32835

zyscoder commented Apr 14, 2020

addaleax commented Apr 14, 2020

zyscoder commented Apr 14, 2020 •

edited

Loading

zyscoder commented Apr 14, 2020

mmarchini commented Apr 14, 2020

zyscoder commented Apr 14, 2020

zyscoder commented Apr 14, 2020

mmarchini commented Apr 14, 2020

addaleax commented Apr 14, 2020

mmarchini commented Apr 14, 2020

zyscoder commented Apr 15, 2020

vdata1 commented Jun 17, 2022 •

edited

Loading

F3n67u commented Jun 17, 2022

Instrumenting nodejs with AddressSanitizer fails resulting of memory leaks #32835

Instrumenting nodejs with AddressSanitizer fails resulting of memory leaks #32835

Comments

zyscoder commented Apr 14, 2020

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior?

What do you see instead?

Additional information

addaleax commented Apr 14, 2020

zyscoder commented Apr 14, 2020 • edited Loading

zyscoder commented Apr 14, 2020

mmarchini commented Apr 14, 2020

zyscoder commented Apr 14, 2020

zyscoder commented Apr 14, 2020

mmarchini commented Apr 14, 2020

addaleax commented Apr 14, 2020

mmarchini commented Apr 14, 2020

zyscoder commented Apr 15, 2020

vdata1 commented Jun 17, 2022 • edited Loading

F3n67u commented Jun 17, 2022

zyscoder commented Apr 14, 2020 •

edited

Loading

vdata1 commented Jun 17, 2022 •

edited

Loading