Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: enable V8's shared read-only heap #42809

Merged
merged 2 commits into from
Apr 24, 2022

Conversation

targos
Copy link
Member

@targos targos commented Apr 21, 2022

It is what V8's build config does by default.

It is what V8's build config does by default.
@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/gyp
  • @nodejs/v8-update

@nodejs-github-bot nodejs-github-bot added build Issues and PRs related to build files or the CI. needs-ci PRs that need a full CI run. tools Issues and PRs related to the tools directory. v8 engine Issues and PRs related to the V8 dependency. labels Apr 21, 2022
@targos
Copy link
Member Author

targos commented Apr 21, 2022

Refs:

node/deps/v8/BUILD.gn

Lines 488 to 491 in fe85cf7

if (v8_enable_shared_ro_heap == "") {
v8_enable_shared_ro_heap = !v8_enable_pointer_compression ||
v8_enable_pointer_compression_shared_cage
}

node/deps/v8/BUILD.gn

Lines 933 to 935 in fe85cf7

if (v8_enable_shared_ro_heap) {
defines += [ "V8_SHARED_RO_HEAP" ]
}

@targos
Copy link
Member Author

targos commented Apr 21, 2022

@nodejs/v8 is this something that we want to do? What would it improve for Node.js?

@targos targos added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 21, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 21, 2022
@nodejs-github-bot
Copy link
Collaborator

@bnoordhuis
Copy link
Member

What would it improve for Node.js?

In theory the shared RO heap speeds up isolate creation and sending objects between them but I don't think it does anything until you start linking isolates together with Isolate::CreateParams::experimental_attach_to_shared_isolate.

@joyeecheung
Copy link
Member

I think it's worth enabling even just to avoid having another configuration not covered by the V8 CI.

@targos
Copy link
Member Author

targos commented Apr 21, 2022

This breaks a test in debug builds: https://ci.nodejs.org/job/node-test-commit-arm-debug/1703/nodes=ubuntu1804-arm64/testReport/junit/(root)/test/parallel_test_worker_nearheaplimit_deadlock/

#
# Fatal error in ../deps/v8/src/heap/read-only-spaces.cc, line 69
# Check failed: read_only_blob_checksum_ == snapshot_checksum (<unprintable> vs. 3337505632).
#
#
#
#FailureMessage Object: 0xffff8cb44588
 1: 0xaaaacb36e4e8 node::DumpBacktrace(_IO_FILE*) [out/Debug/node]
 2: 0xaaaacb52a254  [out/Debug/node]
 3: 0xaaaacb52a280  [out/Debug/node]
 4: 0xaaaaccddd5a4 V8_Fatal(char const*, int, char const*, ...) [out/Debug/node]
 5: 0xaaaacbb46ac4 v8::internal::ReadOnlyArtifacts::VerifyChecksum(v8::internal::SnapshotData*, bool) [out/Debug/node]
 6: 0xaaaacbb4319c v8::internal::ReadOnlyHeap::SetUp(v8::internal::Isolate*, v8::internal::SnapshotData*, bool) [out/Debug/node]
 7: 0xaaaacb9ddb74 v8::internal::Isolate::Init(v8::internal::SnapshotData*, v8::internal::SnapshotData*, v8::internal::SnapshotData*, bool) [out/Debug/node]
 8: 0xaaaacc0c0c0c  [out/Debug/node]
 9: 0xaaaacb7281b8 v8::Isolate::Initialize(v8::Isolate*, v8::Isolate::CreateParams const&) [out/Debug/node]
10: 0xaaaacb59e0f0 node::worker::WorkerThreadData::WorkerThreadData(node::worker::Worker*) [out/Debug/node]
11: 0xaaaacb597b10 node::worker::Worker::Run() [out/Debug/node]
12: 0xaaaacb599adc  [out/Debug/node]
13: 0xaaaacb599b70  [out/Debug/node]
14: 0xffff8f75b088  [/lib/aarch64-linux-gnu/libpthread.so.0]

@joyeecheung
Copy link
Member

@targos That's because the test is intentionally using the default snapshot instead of the Node.js one to make sure that the OOM does not occur during deserialization..so basically enabling shared RO heap breaks --no-node-snapshot in workers (if the main thread is using the embedded snapshot), but since --no-node-snapshot is a internal debug flag that's probably not a big deal. I think adding // Flags: --no-node-snapshot at the top of the test should fix the crash (because then both the main thread and the worker thread will be using the default snapshot)

@joyeecheung
Copy link
Member

Another more solid solution would be to disallow specifying --no-node-snapshot in workers and make sure that they always inherit whatever the main thread uses.

@targos targos added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 22, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 22, 2022
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@targos targos added the commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. label Apr 23, 2022
@targos targos added the commit-queue Add this label to land a pull request using GitHub Actions. label Apr 24, 2022
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Apr 24, 2022
@nodejs-github-bot nodejs-github-bot merged commit daae938 into nodejs:master Apr 24, 2022
@nodejs-github-bot
Copy link
Collaborator

Landed in daae938

@targos targos deleted the shared-ro-heap branch April 24, 2022 06:30
xtx1130 pushed a commit to xtx1130/node that referenced this pull request Apr 25, 2022
It is what V8's build config does by default.

PR-URL: nodejs#42809
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
targos added a commit that referenced this pull request Apr 28, 2022
It is what V8's build config does by default.

PR-URL: #42809
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
nodejs-github-bot pushed a commit that referenced this pull request Apr 28, 2022
We enable the shared read-only heap which currently requires that the
snapshot used in different isolates in the same process to be the same.
Therefore --no-node-snapshot is a per-process option.

PR-URL: #42864
Refs: #42809
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Mohammed Keyvanzadeh <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
targos pushed a commit that referenced this pull request May 2, 2022
We enable the shared read-only heap which currently requires that the
snapshot used in different isolates in the same process to be the same.
Therefore --no-node-snapshot is a per-process option.

PR-URL: #42864
Refs: #42809
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Mohammed Keyvanzadeh <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
@targos targos mentioned this pull request May 2, 2022
codebytere added a commit to electron/electron that referenced this pull request Oct 12, 2022
codebytere added a commit to electron/electron that referenced this pull request Oct 17, 2022
codebytere added a commit to electron/electron that referenced this pull request Oct 19, 2022
codebytere added a commit to electron/electron that referenced this pull request Oct 19, 2022
codebytere added a commit to electron/electron that referenced this pull request Oct 24, 2022
codebytere added a commit to electron/electron that referenced this pull request Nov 8, 2022
codebytere added a commit to electron/electron that referenced this pull request Nov 8, 2022
codebytere added a commit to electron/electron that referenced this pull request Nov 10, 2022
* chore: update to Node.js v18

* child_process: improve argument validation

nodejs/node#41305

* bootstrap: support configure-time user-land snapshot

nodejs/node#42466

* chore: update GN patch

* src: disambiguate terms used to refer to builtins and addons

nodejs/node#44135

* src: use a typed array internally for process._exiting

nodejs/node#43883

* chore: lib/internal/bootstrap -> lib/internal/process

* src: disambiguate terms used to refer to builtins and addons

nodejs/node#44135

* chore: remove redudant browserGlobals patch

* chore: update BoringSSL patch

* src: allow embedder-provided PageAllocator in NodePlatform

nodejs/node#38362

* chore: fixup Node.js crypto tests

- nodejs/node#44171
- nodejs/node#41600

* lib: add Promise methods to avoid-prototype-pollution lint rule

nodejs/node#43849

* deps: update V8 to 10.1

nodejs/node#42657

* src: add kNoBrowserGlobals flag for Environment

nodejs/node#40532

* chore: consolidate asar initialization patches

* deps: update V8 to 10.1

nodejs/node#42657

* deps: update V8 to 9.8

nodejs/node#41610

* src,crypto: remove AllocatedBuffers from crypto_spkac

nodejs/node#40752

* build: enable V8's shared read-only heap

nodejs/node#42809

* src: fix ssize_t error from nghttp2.h

nodejs/node#44393

* chore: fixup ESM patch

* chore: fixup patch indices

* src: merge NativeModuleEnv into NativeModuleLoader

nodejs/node#43824

* [API] Pass OOMDetails to OOMErrorCallback

https://chromium-review.googlesource.com/c/v8/v8/+/3647827

* src: iwyu in cleanup_queue.cc

* src: return Maybe from a couple of functions

nodejs/node#39603

* src: clean up embedder API

nodejs/node#35897

* src: refactor DH groups to delete crypto_groups.h

nodejs/node#43896

* deps,src: use SIMD for normal base64 encoding

nodejs/node#39775

* chore: remove deleted source file

* chore: update patches

* chore: remove deleted source file

* lib: add fetch

nodejs/node#41749

* chore: remove nonexistent node specs

* test: split report OOM tests

nodejs/node#44389

* src: trace fs async api

nodejs/node#44057

* http: trace http request / response

nodejs/node#44102

* test: split test-crypto-dh.js

nodejs/node#40451

* crypto: introduce X509Certificate API

nodejs/node#36804

* src: split property helpers from node::Environment

nodejs/node#44056

* nodejs/node#38905

bootstrap: implement run-time user-land snapshots via --build-snapshot and --snapshot-blob

* lib,src: implement WebAssembly Web API

nodejs/node#42701

* fixup! deps,src: use SIMD for normal base64 encoding

* fixup! src: refactor DH groups to delete crypto_groups.h

* chore: fixup base64 GN file

* fix: check that node::InitializeContext() returns true

* chore: delete _noBrowserGlobals usage

* chore: disable fetch in renderer procceses

* dns: default to verbatim=true in dns.lookup()

nodejs/node#39987

Co-authored-by: PatchUp <73610968+patchup[bot]@users.noreply.github.com>
khalwa pushed a commit to solarwindscloud/electron that referenced this pull request Feb 22, 2023
* chore: update to Node.js v18

* child_process: improve argument validation

nodejs/node#41305

* bootstrap: support configure-time user-land snapshot

nodejs/node#42466

* chore: update GN patch

* src: disambiguate terms used to refer to builtins and addons

nodejs/node#44135

* src: use a typed array internally for process._exiting

nodejs/node#43883

* chore: lib/internal/bootstrap -> lib/internal/process

* src: disambiguate terms used to refer to builtins and addons

nodejs/node#44135

* chore: remove redudant browserGlobals patch

* chore: update BoringSSL patch

* src: allow embedder-provided PageAllocator in NodePlatform

nodejs/node#38362

* chore: fixup Node.js crypto tests

- nodejs/node#44171
- nodejs/node#41600

* lib: add Promise methods to avoid-prototype-pollution lint rule

nodejs/node#43849

* deps: update V8 to 10.1

nodejs/node#42657

* src: add kNoBrowserGlobals flag for Environment

nodejs/node#40532

* chore: consolidate asar initialization patches

* deps: update V8 to 10.1

nodejs/node#42657

* deps: update V8 to 9.8

nodejs/node#41610

* src,crypto: remove AllocatedBuffers from crypto_spkac

nodejs/node#40752

* build: enable V8's shared read-only heap

nodejs/node#42809

* src: fix ssize_t error from nghttp2.h

nodejs/node#44393

* chore: fixup ESM patch

* chore: fixup patch indices

* src: merge NativeModuleEnv into NativeModuleLoader

nodejs/node#43824

* [API] Pass OOMDetails to OOMErrorCallback

https://chromium-review.googlesource.com/c/v8/v8/+/3647827

* src: iwyu in cleanup_queue.cc

* src: return Maybe from a couple of functions

nodejs/node#39603

* src: clean up embedder API

nodejs/node#35897

* src: refactor DH groups to delete crypto_groups.h

nodejs/node#43896

* deps,src: use SIMD for normal base64 encoding

nodejs/node#39775

* chore: remove deleted source file

* chore: update patches

* chore: remove deleted source file

* lib: add fetch

nodejs/node#41749

* chore: remove nonexistent node specs

* test: split report OOM tests

nodejs/node#44389

* src: trace fs async api

nodejs/node#44057

* http: trace http request / response

nodejs/node#44102

* test: split test-crypto-dh.js

nodejs/node#40451

* crypto: introduce X509Certificate API

nodejs/node#36804

* src: split property helpers from node::Environment

nodejs/node#44056

* nodejs/node#38905

bootstrap: implement run-time user-land snapshots via --build-snapshot and --snapshot-blob

* lib,src: implement WebAssembly Web API

nodejs/node#42701

* fixup! deps,src: use SIMD for normal base64 encoding

* fixup! src: refactor DH groups to delete crypto_groups.h

* chore: fixup base64 GN file

* fix: check that node::InitializeContext() returns true

* chore: delete _noBrowserGlobals usage

* chore: disable fetch in renderer procceses

* dns: default to verbatim=true in dns.lookup()

nodejs/node#39987

Co-authored-by: PatchUp <73610968+patchup[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Issues and PRs related to build files or the CI. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. needs-ci PRs that need a full CI run. tools Issues and PRs related to the tools directory. v8 engine Issues and PRs related to the V8 dependency.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants