Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segmentation fault in worker test #115

Closed
targos opened this issue Sep 11, 2019 · 15 comments
Closed

Segmentation fault in worker test #115

targos opened this issue Sep 11, 2019 · 15 comments

Comments

@targos
Copy link
Member

targos commented Sep 11, 2019

On canary-base, there is currently one test failure:

out/Release/node test/parallel/test-worker-sharedarraybuffer-from-worker-thread.js        
[1]    11828 segmentation fault (core dumped)  out/Release/node

Backtrace:

#0  0x0000000000000020 in ?? ()
#1  0x0000000000e7a9f9 in v8::internal::BackingStore::~BackingStore() ()
#2  0x0000000000ba6dab in std::_Sp_counted_deleter<v8::internal::BackingStore*, std::default_delete<v8::internal::BackingStore>, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() ()
#3  0x0000000000d11a51 in v8::internal::ArrayBufferTracker::FreeAll(v8::internal::Page*) ()
#4  0x0000000000d11b90 in v8::internal::ArrayBufferTracker::TearDown(v8::internal::Heap*) ()
#5  0x0000000000d60b8c in v8::internal::Heap::TearDown() ()
#6  0x0000000000cfb6e3 in v8::internal::Isolate::Deinit() ()
#7  0x0000000000cfd57e in v8::internal::Isolate::Delete(v8::internal::Isolate*)
    ()
#8  0x0000000000a5c209 in node::NodeMainInstance::~NodeMainInstance() ()
#9  0x00000000009e738b in node::Start(int, char**) ()
#10 0x00007f81dba05f33 in __libc_start_main () from /lib64/libc.so.6
#11 0x0000000000951abe in _start ()

/cc @addaleax

@targos
Copy link
Member Author

targos commented Sep 11, 2019

Backtrace with debug build:

#0  0x000000000156b765 in v8::internal::BackingStore::~BackingStore (this=0x7f523813abf0, __in_chrg=<optimized out>) at ../../deps/v8/src/objects/backing-store.cc:157
#1  0x00000000010d8102 in std::default_delete<v8::internal::BackingStore>::operator() (this=<optimized out>, __ptr=0x7f523813abf0) at /usr/include/c++/9/bits/unique_ptr.h:75
#2  std::default_delete<v8::internal::BackingStore>::operator() (this=<optimized out>, __ptr=0x7f523813abf0) at /usr/include/c++/9/bits/unique_ptr.h:75
#3  std::_Sp_counted_deleter<v8::internal::BackingStore*, std::default_delete<v8::internal::BackingStore>, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_dispose (
    this=<optimized out>) at /usr/include/c++/9/bits/shared_ptr_base.h:471
#4  0x0000000001381a55 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x7f5238121cb0) at /usr/include/c++/9/ext/atomicity.h:69
#5  std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x7f5238121cb0) at /usr/include/c++/9/bits/shared_ptr_base.h:148
#6  std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/9/bits/shared_ptr_base.h:730
#7  std::__shared_ptr<v8::internal::BackingStore, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr (this=<optimized out>, __in_chrg=<optimized out>)
    at /usr/include/c++/9/bits/shared_ptr_base.h:1169
#8  std::shared_ptr<v8::internal::BackingStore>::~shared_ptr (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/9/bits/shared_ptr.h:103
#9  std::pair<v8::internal::JSArrayBuffer const, std::shared_ptr<v8::internal::BackingStore> >::~pair (this=<optimized out>, __in_chrg=<optimized out>)
    at /usr/include/c++/9/bits/stl_pair.h:208
#10 __gnu_cxx::new_allocator<std::__detail::_Hash_node<std::pair<v8::internal::JSArrayBuffer const, std::shared_ptr<v8::internal::BackingStore> >, true> >::destroy<std::pair<v8::internal::JSArrayBuffer const, std::shared_ptr<v8::internal::BackingStore> > > (this=<optimized out>, __p=<optimized out>) at /usr/include/c++/9/ext/new_allocator.h:153
#11 std::allocator_traits<std::allocator<std::__detail::_Hash_node<std::pair<v8::internal::JSArrayBuffer const, std::shared_ptr<v8::internal::BackingStore> >, true> > >::destroy<std::pair<v8::internal::JSArrayBuffer const, std::shared_ptr<v8::internal::BackingStore> > > (__a=..., __p=<optimized out>) at /usr/include/c++/9/bits/alloc_traits.h:497
#12 std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<v8::internal::JSArrayBuffer const, std::shared_ptr<v8::internal::BackingStore> >, true> > >::_M_deallocate_node (this=<optimized out>, __n=<optimized out>) at /usr/include/c++/9/bits/hashtable_policy.h:2102
#13 std::_Hashtable<v8::internal::JSArrayBuffer, std::pair<v8::internal::JSArrayBuffer const, std::shared_ptr<v8::internal::BackingStore> >, std::allocator<std::pair<v8::internal::JSArrayBuffer const, std::shared_ptr<v8::internal::BackingStore> > >, std::__detail::_Select1st, std::equal_to<v8::internal::JSArrayBuffer>, v8::internal::LocalArrayBufferTracker::Hasher, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_erase (__n=<optimized out>, 
    __prev_n=<optimized out>, __bkt=<optimized out>, this=<optimized out>) at /usr/include/c++/9/bits/hashtable.h:1886
#14 std::_Hashtable<v8::internal::JSArrayBuffer, std::pair<v8::internal::JSArrayBuffer const, std::shared_ptr<v8::internal::BackingStore> >, std::allocator<std::pair<v8::internal::JSArrayBuffer const, std::shared_ptr<v8::internal::BackingStore> > >, std::__detail::_Select1st, std::equal_to<v8::internal::JSArrayBuffer>, v8::internal::LocalArrayBufferTracker::Hasher, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::erase (__it=..., 
    this=<optimized out>) at /usr/include/c++/9/bits/hashtable.h:1861
#15 std::_Hashtable<v8::internal::JSArrayBuffer, std::pair<v8::internal::JSArrayBuffer const, std::shared_ptr<v8::internal::BackingStore> >, std::allocator<std::pair<v8::internal::JSArrayBuffer const, std::shared_ptr<v8::internal::BackingStore> > >, std::__detail::_Select1st, std::equal_to<v8::internal::JSArrayBuffer>, v8::internal::LocalArrayBufferTracker::Hasher, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::erase (__it=..., 
    this=<optimized out>) at /usr/include/c++/9/bits/hashtable.h:768
#16 std::unordered_map<v8::internal::JSArrayBuffer, std::shared_ptr<v8::internal::BackingStore>, v8::internal::LocalArrayBufferTracker::Hasher, std::equal_to<v8::internal::JSArrayBuffer>, std::allocator<std::pair<v8::internal::JSArrayBuffer const, std::shared_ptr<v8::internal::BackingStore> > > >::erase (__position=..., this=<optimized out>)
    at /usr/include/c++/9/bits/unordered_map.h:798
#17 v8::internal::LocalArrayBufferTracker::Free<v8::internal::ArrayBufferTracker::FreeAll(v8::internal::Page*)::<lambda(v8::internal::JSArrayBuffer)> > (this=<optimized out>, 
    should_free=...) at ../../deps/v8/src/heap/array-buffer-tracker-inl.h:106
#18 v8::internal::ArrayBufferTracker::FreeAll (page=page@entry=0x3a1677780000) at ../../deps/v8/src/heap/array-buffer-tracker.cc:106
#19 0x0000000001381d38 in v8::internal::ArrayBufferTracker::TearDown (heap=heap@entry=0x40f27c0) at ../../deps/v8/src/heap/array-buffer-tracker.cc:148
#20 0x00000000013fb656 in v8::internal::Heap::TearDown (this=0x40f27c0) at ../../deps/v8/src/heap/heap.cc:5286
#21 0x000000000135c857 in v8::internal::Isolate::Deinit (this=0x40e94a0) at ../../deps/v8/src/execution/isolate.cc:3029
#22 0x000000000135eb55 in v8::internal::Isolate::Delete (isolate=0x40e94a0) at ../../deps/v8/src/execution/isolate.cc:2850
#23 0x0000000000f30e67 in node::NodeMainInstance::~NodeMainInstance (this=0x7fffbb09f010, __in_chrg=<optimized out>) at ../../src/node_main_instance.cc:91
#24 0x0000000000e8e56e in node::Start (argc=2, argv=0x7fffbb09f298) at ../../src/node.cc:1077
#25 0x00000000024dbd02 in main (argc=2, argv=0x7fffbb09f298) at ../../src/node_main.cc:126
(gdb) info registers
rax            0x0                 0
rbx            0x7f523813abf0      139991104859120
rcx            0x0                 0
rdx            0x2                 2
rsi            0x0                 0
rdi            0x423a820           69445664
rbp            0x7f523813abf8      0x7f523813abf8
rsp            0x7fffbb09ee80      0x7fffbb09ee80
r8             0x3c296e8           63084264
r9             0x0                 0
r10            0x78                120
r11            0x8                 8
r12            0x423a820           69445664
r13            0x0                 0
r14            0x4                 4
r15            0x3a1677780000      63868168044544
rip            0x156b765           0x156b765 <v8::internal::BackingStore::~BackingStore()+325>
eflags         0x10246             [ PF ZF IF RF ]
cs             0x33                51
ss             0x2b                43
ds             0x0                 0
es             0x0                 0
fs             0x0                 0
gs             0x0                 0

@addaleax
Copy link
Member

Thanks for the ping, I’ll look into it

@addaleax
Copy link
Member

I think this is v8/v8@b6b7de0 … since there are no accompanying API changes in V8, I think code that worked before should also work after, so this might be a bug in V8? I’ll still try to figure out what’s happening here exactly, but I think there might be a chance that V8 is trying to free an externalized SharedArrayBuffer instance on Isolate teardown.

Is Node.js still part of the V8 CI? https://github.com/v8/node/ doesn’t seem up to date at all… /cc @hashseed @titzer

@targos
Copy link
Member Author

targos commented Sep 12, 2019

Here is the node V8 CI run for that commit: https://ci.chromium.org/p/node-ci/builders/try/node_ci_linux64_rel/b8906314882042544400
It is green but I don't see this test in the output.

@targos
Copy link
Member Author

targos commented Sep 12, 2019

Of course, you added the test less than a month ago, so without an up to date v8/node repo it won't be executed by the V8 CI

@addaleax
Copy link
Member

I think there might be a chance that V8 is trying to free an externalized SharedArrayBuffer instance on Isolate teardown.

Yes, this is it. It might be possible to work around this in Node.js, but the new semantics aren’t really clear to me at a glance.

@addaleax
Copy link
Member

Hm yeah, I’m a bit stumped. The V8 change requires the ArrayBuffer::Allocator behind a SharedArrayBuffer to have a longer lifetime than previously, and while the change is one in the right direction, the different timing means that V8 tries to access it after we destroyed it.

@addaleax
Copy link
Member

So … what we could do, I think, is to store all ArrayBuffer::Allocator instances referenced by a Node.js Environment in a separate array that outlives the Environment, and then release those after the Isolate has been torn down – this isn’t a good solution, though, and one that requires a breaking change to our embedder API, at least temporarily (if we want to avoid memory leaks).

@targos What Node.js release lines would the V8 change affect? Is it something that would align with a semver-major release on our side?

@targos
Copy link
Member Author

targos commented Sep 13, 2019

This change won't land on time for Node.js 13.0.0.
There are at least two options:

  • We make the necessary breaking API changes before 13.0.0, allowing us to upgrade V8 in a semver-minor when the version containing this change is released.
  • We wait for the integration of this change in Node.js to adapt our API, locking us to the previous V8 version until Node.js 14.0.0 is released.

addaleax added a commit to addaleax/node that referenced this issue Sep 20, 2019
Keep the `ArrayBuffer::Allocator` behind a `SharedArrayBuffer` instance
alive for at least as long as the receiving Isolate lives, if the
`SharedArrayBuffer` instance isn’t already destroyed through GC.

This is to work around the fact that V8 7.9 started refactoring
how backing stores for `SharedArrayBuffer` instances work, changing
the timing of the call that releases the backing store to be
during Isolate disposal.

The flag added to the test is optional but helps verify that the
backing store is actually free’d at the end of the test and does not
leak memory.

Fixes: nodejs/node-v8#115
@addaleax
Copy link
Member

@targos Sometimes it’s good to get a bit of distance and think about it again :) Found a way to address this without API changes: nodejs/node#29637

targos pushed a commit to nodejs/node that referenced this issue Oct 1, 2019
Keep the `ArrayBuffer::Allocator` behind a `SharedArrayBuffer` instance
alive for at least as long as the receiving Isolate lives, if the
`SharedArrayBuffer` instance isn’t already destroyed through GC.

This is to work around the fact that V8 7.9 started refactoring
how backing stores for `SharedArrayBuffer` instances work, changing
the timing of the call that releases the backing store to be
during Isolate disposal.

The flag added to the test is optional but helps verify that the
backing store is actually free’d at the end of the test and does not
leak memory.

Fixes: nodejs/node-v8#115
PR-URL: #29637
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Ben Noordhuis <[email protected]>
addaleax added a commit to addaleax/node that referenced this issue Oct 20, 2019
This test would be broken by V8 7.9 due to the changed `ArrayBuffer`
backing store management (the same way that V8 7.8 broke this for
`SharedArrayBuffer`s). While working on a solution, it would be
good to already have this test in Node.js to avoid unnecessary
accidental breakage.

Refs: nodejs/node-v8#115
Trott pushed a commit to nodejs/node that referenced this issue Oct 22, 2019
This test would be broken by V8 7.9 due to the changed `ArrayBuffer`
backing store management (the same way that V8 7.8 broke this for
`SharedArrayBuffer`s). While working on a solution, it would be
good to already have this test in Node.js to avoid unnecessary
accidental breakage.

Refs: nodejs/node-v8#115

PR-URL: #30044
Reviewed-By: Yongsheng Zhang <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
MylesBorins pushed a commit to nodejs/node that referenced this issue Oct 23, 2019
This test would be broken by V8 7.9 due to the changed `ArrayBuffer`
backing store management (the same way that V8 7.8 broke this for
`SharedArrayBuffer`s). While working on a solution, it would be
good to already have this test in Node.js to avoid unnecessary
accidental breakage.

Refs: nodejs/node-v8#115

PR-URL: #30044
Reviewed-By: Yongsheng Zhang <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
MylesBorins pushed a commit to nodejs/node that referenced this issue Oct 23, 2019
This test would be broken by V8 7.9 due to the changed `ArrayBuffer`
backing store management (the same way that V8 7.8 broke this for
`SharedArrayBuffer`s). While working on a solution, it would be
good to already have this test in Node.js to avoid unnecessary
accidental breakage.

Refs: nodejs/node-v8#115

PR-URL: #30044
Reviewed-By: Yongsheng Zhang <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
pull bot pushed a commit to ksti/v8 that referenced this issue Oct 24, 2019
Add an `array_buffer_allocator_shared` field to the
`Isolate::CreateParams` struct that allows embedders to share
ownership of the ArrayBuffer::Allocator with V8, and which in
particular means that when this method is used that the
BackingStore deleter will not perform an use-after-free access to the
Allocator under certain circumstances.

For Background:

tl;dr: This is necessary for Node.js to perform the transition to
V8 7.9, because of the way that ArrayBuffer::Allocators and their
lifetimes currently work there.

In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
Changing that would currently be impractical, as each allocator
depends on per-Isolate state. However, now that backing stores
are managed globally and keep a pointer to the original
ArrayBuffer::Allocator, this means that when transferring an
ArrayBuffer (e.g. from one Worker to another through postMessage()),
the original Allocator has to be kept alive until the ArrayBuffer
no longer exists in the receiving Isolate (or until that Isolate
is disposed). See [1] for an example Node.js test that fails with
V8 7.9.

This problem also existed for SharedArrayBuffers, where Node.js
was broken by V8 earlier for the same reasons (see [2] for the bug
report on that and [3] for the resolution in Node.js).
For SharedArrayBuffers, we already had extensive tracking logic,
so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
was not a significant amount of work. However, the mechanism for
transferring non-shared ArrayBuffers is quite different, and
it seems both easier for us and better for V8 from an API standpoint
to keep the Allocator alive from where it is being referenced.

By sharing memory with the custom deleter function/data pair,
this comes at no memory overhead.

[1]: nodejs/node#30044
[2]: nodejs/node-v8#115
[3]: nodejs/node#29637

Bug: v8:9380
Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
Commit-Queue: Ulan Degenbaev <[email protected]>
Reviewed-by: Ulan Degenbaev <[email protected]>
Reviewed-by: Igor Sheludko <[email protected]>
Cr-Commit-Position: refs/heads/master@{#64542}
targos added a commit to targos/node that referenced this issue Oct 24, 2019
Original commit message:

    [api] Add possibility for BackingStore to keep Allocator alive

    Add an `array_buffer_allocator_shared` field to the
    `Isolate::CreateParams` struct that allows embedders to share
    ownership of the ArrayBuffer::Allocator with V8, and which in
    particular means that when this method is used that the
    BackingStore deleter will not perform an use-after-free access to the
    Allocator under certain circumstances.

    For Background:

    tl;dr: This is necessary for Node.js to perform the transition to
    V8 7.9, because of the way that ArrayBuffer::Allocators and their
    lifetimes currently work there.

    In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
    Changing that would currently be impractical, as each allocator
    depends on per-Isolate state. However, now that backing stores
    are managed globally and keep a pointer to the original
    ArrayBuffer::Allocator, this means that when transferring an
    ArrayBuffer (e.g. from one Worker to another through postMessage()),
    the original Allocator has to be kept alive until the ArrayBuffer
    no longer exists in the receiving Isolate (or until that Isolate
    is disposed). See [1] for an example Node.js test that fails with
    V8 7.9.

    This problem also existed for SharedArrayBuffers, where Node.js
    was broken by V8 earlier for the same reasons (see [2] for the bug
    report on that and [3] for the resolution in Node.js).
    For SharedArrayBuffers, we already had extensive tracking logic,
    so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
    was not a significant amount of work. However, the mechanism for
    transferring non-shared ArrayBuffers is quite different, and
    it seems both easier for us and better for V8 from an API standpoint
    to keep the Allocator alive from where it is being referenced.

    By sharing memory with the custom deleter function/data pair,
    this comes at no memory overhead.

    [1]: nodejs#30044
    [2]: nodejs/node-v8#115
    [3]: nodejs#29637

    Bug: v8:9380
    Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
    Commit-Queue: Ulan Degenbaev <[email protected]>
    Reviewed-by: Ulan Degenbaev <[email protected]>
    Reviewed-by: Igor Sheludko <[email protected]>
    Cr-Commit-Position: refs/heads/master@{#64542}

Refs: v8/v8@6b0a953
targos added a commit to targos/node that referenced this issue Oct 24, 2019
Original commit message:

    [api] Add possibility for BackingStore to keep Allocator alive

    Add an `array_buffer_allocator_shared` field to the
    `Isolate::CreateParams` struct that allows embedders to share
    ownership of the ArrayBuffer::Allocator with V8, and which in
    particular means that when this method is used that the
    BackingStore deleter will not perform an use-after-free access to the
    Allocator under certain circumstances.

    For Background:

    tl;dr: This is necessary for Node.js to perform the transition to
    V8 7.9, because of the way that ArrayBuffer::Allocators and their
    lifetimes currently work there.

    In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
    Changing that would currently be impractical, as each allocator
    depends on per-Isolate state. However, now that backing stores
    are managed globally and keep a pointer to the original
    ArrayBuffer::Allocator, this means that when transferring an
    ArrayBuffer (e.g. from one Worker to another through postMessage()),
    the original Allocator has to be kept alive until the ArrayBuffer
    no longer exists in the receiving Isolate (or until that Isolate
    is disposed). See [1] for an example Node.js test that fails with
    V8 7.9.

    This problem also existed for SharedArrayBuffers, where Node.js
    was broken by V8 earlier for the same reasons (see [2] for the bug
    report on that and [3] for the resolution in Node.js).
    For SharedArrayBuffers, we already had extensive tracking logic,
    so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
    was not a significant amount of work. However, the mechanism for
    transferring non-shared ArrayBuffers is quite different, and
    it seems both easier for us and better for V8 from an API standpoint
    to keep the Allocator alive from where it is being referenced.

    By sharing memory with the custom deleter function/data pair,
    this comes at no memory overhead.

    [1]: nodejs#30044
    [2]: nodejs/node-v8#115
    [3]: nodejs#29637

    Bug: v8:9380
    Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
    Commit-Queue: Ulan Degenbaev <[email protected]>
    Reviewed-by: Ulan Degenbaev <[email protected]>
    Reviewed-by: Igor Sheludko <[email protected]>
    Cr-Commit-Position: refs/heads/master@{#64542}

Refs: v8/v8@6b0a953
targos added a commit to targos/node that referenced this issue Oct 28, 2019
Original commit message:

    [api] Add possibility for BackingStore to keep Allocator alive

    Add an `array_buffer_allocator_shared` field to the
    `Isolate::CreateParams` struct that allows embedders to share
    ownership of the ArrayBuffer::Allocator with V8, and which in
    particular means that when this method is used that the
    BackingStore deleter will not perform an use-after-free access to the
    Allocator under certain circumstances.

    For Background:

    tl;dr: This is necessary for Node.js to perform the transition to
    V8 7.9, because of the way that ArrayBuffer::Allocators and their
    lifetimes currently work there.

    In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
    Changing that would currently be impractical, as each allocator
    depends on per-Isolate state. However, now that backing stores
    are managed globally and keep a pointer to the original
    ArrayBuffer::Allocator, this means that when transferring an
    ArrayBuffer (e.g. from one Worker to another through postMessage()),
    the original Allocator has to be kept alive until the ArrayBuffer
    no longer exists in the receiving Isolate (or until that Isolate
    is disposed). See [1] for an example Node.js test that fails with
    V8 7.9.

    This problem also existed for SharedArrayBuffers, where Node.js
    was broken by V8 earlier for the same reasons (see [2] for the bug
    report on that and [3] for the resolution in Node.js).
    For SharedArrayBuffers, we already had extensive tracking logic,
    so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
    was not a significant amount of work. However, the mechanism for
    transferring non-shared ArrayBuffers is quite different, and
    it seems both easier for us and better for V8 from an API standpoint
    to keep the Allocator alive from where it is being referenced.

    By sharing memory with the custom deleter function/data pair,
    this comes at no memory overhead.

    [1]: nodejs#30044
    [2]: nodejs/node-v8#115
    [3]: nodejs#29637

    Bug: v8:9380
    Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
    Commit-Queue: Ulan Degenbaev <[email protected]>
    Reviewed-by: Ulan Degenbaev <[email protected]>
    Reviewed-by: Igor Sheludko <[email protected]>
    Cr-Commit-Position: refs/heads/master@{#64542}

Refs: v8/v8@6b0a953
targos added a commit to targos/node that referenced this issue Nov 1, 2019
Original commit message:

    [api] Add possibility for BackingStore to keep Allocator alive

    Add an `array_buffer_allocator_shared` field to the
    `Isolate::CreateParams` struct that allows embedders to share
    ownership of the ArrayBuffer::Allocator with V8, and which in
    particular means that when this method is used that the
    BackingStore deleter will not perform an use-after-free access to the
    Allocator under certain circumstances.

    For Background:

    tl;dr: This is necessary for Node.js to perform the transition to
    V8 7.9, because of the way that ArrayBuffer::Allocators and their
    lifetimes currently work there.

    In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
    Changing that would currently be impractical, as each allocator
    depends on per-Isolate state. However, now that backing stores
    are managed globally and keep a pointer to the original
    ArrayBuffer::Allocator, this means that when transferring an
    ArrayBuffer (e.g. from one Worker to another through postMessage()),
    the original Allocator has to be kept alive until the ArrayBuffer
    no longer exists in the receiving Isolate (or until that Isolate
    is disposed). See [1] for an example Node.js test that fails with
    V8 7.9.

    This problem also existed for SharedArrayBuffers, where Node.js
    was broken by V8 earlier for the same reasons (see [2] for the bug
    report on that and [3] for the resolution in Node.js).
    For SharedArrayBuffers, we already had extensive tracking logic,
    so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
    was not a significant amount of work. However, the mechanism for
    transferring non-shared ArrayBuffers is quite different, and
    it seems both easier for us and better for V8 from an API standpoint
    to keep the Allocator alive from where it is being referenced.

    By sharing memory with the custom deleter function/data pair,
    this comes at no memory overhead.

    [1]: nodejs#30044
    [2]: nodejs/node-v8#115
    [3]: nodejs#29637

    Bug: v8:9380
    Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
    Commit-Queue: Ulan Degenbaev <[email protected]>
    Reviewed-by: Ulan Degenbaev <[email protected]>
    Reviewed-by: Igor Sheludko <[email protected]>
    Cr-Commit-Position: refs/heads/master@{#64542}

Refs: v8/v8@6b0a953
targos pushed a commit to nodejs/node that referenced this issue Nov 8, 2019
This test would be broken by V8 7.9 due to the changed `ArrayBuffer`
backing store management (the same way that V8 7.8 broke this for
`SharedArrayBuffer`s). While working on a solution, it would be
good to already have this test in Node.js to avoid unnecessary
accidental breakage.

Refs: nodejs/node-v8#115

PR-URL: #30044
Reviewed-By: Yongsheng Zhang <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
targos added a commit to targos/node that referenced this issue Nov 8, 2019
Original commit message:

    [api] Add possibility for BackingStore to keep Allocator alive

    Add an `array_buffer_allocator_shared` field to the
    `Isolate::CreateParams` struct that allows embedders to share
    ownership of the ArrayBuffer::Allocator with V8, and which in
    particular means that when this method is used that the
    BackingStore deleter will not perform an use-after-free access to the
    Allocator under certain circumstances.

    For Background:

    tl;dr: This is necessary for Node.js to perform the transition to
    V8 7.9, because of the way that ArrayBuffer::Allocators and their
    lifetimes currently work there.

    In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
    Changing that would currently be impractical, as each allocator
    depends on per-Isolate state. However, now that backing stores
    are managed globally and keep a pointer to the original
    ArrayBuffer::Allocator, this means that when transferring an
    ArrayBuffer (e.g. from one Worker to another through postMessage()),
    the original Allocator has to be kept alive until the ArrayBuffer
    no longer exists in the receiving Isolate (or until that Isolate
    is disposed). See [1] for an example Node.js test that fails with
    V8 7.9.

    This problem also existed for SharedArrayBuffers, where Node.js
    was broken by V8 earlier for the same reasons (see [2] for the bug
    report on that and [3] for the resolution in Node.js).
    For SharedArrayBuffers, we already had extensive tracking logic,
    so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
    was not a significant amount of work. However, the mechanism for
    transferring non-shared ArrayBuffers is quite different, and
    it seems both easier for us and better for V8 from an API standpoint
    to keep the Allocator alive from where it is being referenced.

    By sharing memory with the custom deleter function/data pair,
    this comes at no memory overhead.

    [1]: nodejs#30044
    [2]: nodejs/node-v8#115
    [3]: nodejs#29637

    Bug: v8:9380
    Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
    Commit-Queue: Ulan Degenbaev <[email protected]>
    Reviewed-by: Ulan Degenbaev <[email protected]>
    Reviewed-by: Igor Sheludko <[email protected]>
    Cr-Commit-Position: refs/heads/master@{#64542}

Refs: v8/v8@6b0a953

PR-URL: nodejs#30020
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Jiawen Geng <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
targos pushed a commit to nodejs/node that referenced this issue Nov 10, 2019
This test would be broken by V8 7.9 due to the changed `ArrayBuffer`
backing store management (the same way that V8 7.8 broke this for
`SharedArrayBuffer`s). While working on a solution, it would be
good to already have this test in Node.js to avoid unnecessary
accidental breakage.

Refs: nodejs/node-v8#115

PR-URL: #30044
Reviewed-By: Yongsheng Zhang <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
targos pushed a commit to nodejs/node that referenced this issue Nov 11, 2019
This test would be broken by V8 7.9 due to the changed `ArrayBuffer`
backing store management (the same way that V8 7.8 broke this for
`SharedArrayBuffer`s). While working on a solution, it would be
good to already have this test in Node.js to avoid unnecessary
accidental breakage.

Refs: nodejs/node-v8#115

PR-URL: #30044
Reviewed-By: Yongsheng Zhang <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
targos added a commit to targos/node that referenced this issue Nov 17, 2019
Original commit message:

    [api] Add possibility for BackingStore to keep Allocator alive

    Add an `array_buffer_allocator_shared` field to the
    `Isolate::CreateParams` struct that allows embedders to share
    ownership of the ArrayBuffer::Allocator with V8, and which in
    particular means that when this method is used that the
    BackingStore deleter will not perform an use-after-free access to the
    Allocator under certain circumstances.

    For Background:

    tl;dr: This is necessary for Node.js to perform the transition to
    V8 7.9, because of the way that ArrayBuffer::Allocators and their
    lifetimes currently work there.

    In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
    Changing that would currently be impractical, as each allocator
    depends on per-Isolate state. However, now that backing stores
    are managed globally and keep a pointer to the original
    ArrayBuffer::Allocator, this means that when transferring an
    ArrayBuffer (e.g. from one Worker to another through postMessage()),
    the original Allocator has to be kept alive until the ArrayBuffer
    no longer exists in the receiving Isolate (or until that Isolate
    is disposed). See [1] for an example Node.js test that fails with
    V8 7.9.

    This problem also existed for SharedArrayBuffers, where Node.js
    was broken by V8 earlier for the same reasons (see [2] for the bug
    report on that and [3] for the resolution in Node.js).
    For SharedArrayBuffers, we already had extensive tracking logic,
    so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
    was not a significant amount of work. However, the mechanism for
    transferring non-shared ArrayBuffers is quite different, and
    it seems both easier for us and better for V8 from an API standpoint
    to keep the Allocator alive from where it is being referenced.

    By sharing memory with the custom deleter function/data pair,
    this comes at no memory overhead.

    [1]: nodejs#30044
    [2]: nodejs/node-v8#115
    [3]: nodejs#29637

    Bug: v8:9380
    Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
    Commit-Queue: Ulan Degenbaev <[email protected]>
    Reviewed-by: Ulan Degenbaev <[email protected]>
    Reviewed-by: Igor Sheludko <[email protected]>
    Cr-Commit-Position: refs/heads/master@{#64542}

Refs: v8/v8@6b0a953

PR-URL: nodejs#30020
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Jiawen Geng <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
MylesBorins pushed a commit to nodejs/node that referenced this issue Nov 21, 2019
Original commit message:

    [api] Add possibility for BackingStore to keep Allocator alive

    Add an `array_buffer_allocator_shared` field to the
    `Isolate::CreateParams` struct that allows embedders to share
    ownership of the ArrayBuffer::Allocator with V8, and which in
    particular means that when this method is used that the
    BackingStore deleter will not perform an use-after-free access to the
    Allocator under certain circumstances.

    For Background:

    tl;dr: This is necessary for Node.js to perform the transition to
    V8 7.9, because of the way that ArrayBuffer::Allocators and their
    lifetimes currently work there.

    In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
    Changing that would currently be impractical, as each allocator
    depends on per-Isolate state. However, now that backing stores
    are managed globally and keep a pointer to the original
    ArrayBuffer::Allocator, this means that when transferring an
    ArrayBuffer (e.g. from one Worker to another through postMessage()),
    the original Allocator has to be kept alive until the ArrayBuffer
    no longer exists in the receiving Isolate (or until that Isolate
    is disposed). See [1] for an example Node.js test that fails with
    V8 7.9.

    This problem also existed for SharedArrayBuffers, where Node.js
    was broken by V8 earlier for the same reasons (see [2] for the bug
    report on that and [3] for the resolution in Node.js).
    For SharedArrayBuffers, we already had extensive tracking logic,
    so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
    was not a significant amount of work. However, the mechanism for
    transferring non-shared ArrayBuffers is quite different, and
    it seems both easier for us and better for V8 from an API standpoint
    to keep the Allocator alive from where it is being referenced.

    By sharing memory with the custom deleter function/data pair,
    this comes at no memory overhead.

    [1]: #30044
    [2]: nodejs/node-v8#115
    [3]: #29637

    Bug: v8:9380
    Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
    Commit-Queue: Ulan Degenbaev <[email protected]>
    Reviewed-by: Ulan Degenbaev <[email protected]>
    Reviewed-by: Igor Sheludko <[email protected]>
    Cr-Commit-Position: refs/heads/master@{#64542}

Refs: v8/v8@6b0a953

Backport-PR-URL: #30513
PR-URL: #30020
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Jiawen Geng <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
@d3x0r
Copy link

d3x0r commented Jun 8, 2020

Is this still an issue? I ask, because I've updated to backing store in Node 14 to solve the deprecation warnings; everything's kind of working, but the backing store is created... in not the main thread... and is being released in not the main thread; and is crashing in the deconstructor method of BackingStore...

@JingkunHu
Copy link

Segmentation fault still happens when running node 12.18.4 with multi-worker-threads

@Trott
Copy link
Member

Trott commented Oct 3, 2020

Segmentation fault still happens when running node 12.18.4 with multi-worker-threads

I don't believe this was fixed in 12.x. Fixed in 14.0.0 though.

@d3x0r
Copy link

d3x0r commented Oct 4, 2020

No, in 14, windows debug addons crash with release Node... have to build a debug node to use debug plugins, otherwise the arraybuffer delete uses the wrong runtime (uses the delete linked with the addon, and not the delete linked with node that created the object in the first place)
Edit:err, sorry, I guess 'this' doesn't fail... just backing stores.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants