Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

src: make --use-largepages a runtime option #30954

Closed

Conversation

gabrielschulhof
Copy link
Contributor

Moves the option that instructs Node.js to-remap its static code to
large pages from a configure-time option to a runtime option. This
should make it easy to assess the performance impact of such a change
without having to custom-build.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • documentation is changed or added
  • commit message follows commit guidelines

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. labels Dec 13, 2019
Copy link
Member

@addaleax addaleax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a basic test that e.g. just spawns node --use-largepages -p 42 and makes sure that the output is correct (ideally verify both stderr and stdout)? That might give us a better idea under which circumstances this is or is not working

doc/api/cli.md Outdated Show resolved Hide resolved
Copy link
Member

@bnoordhuis bnoordhuis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but can you also update doc/node.1 and can I suggest applying this patch so the build won't break on older Linux systems?

diff --git a/src/large_pages/node_large_page.cc b/src/large_pages/node_large_page.cc
index 68fa178b40..d13af4c11e 100644
--- a/src/large_pages/node_large_page.cc
+++ b/src/large_pages/node_large_page.cc
@@ -352,7 +352,7 @@ MoveTextRegionToLargePages(const text_region& r) {
     return -1;
   }
 
-  ret = madvise(tmem, size, MADV_HUGEPAGE);
+  ret = madvise(tmem, size, 14 /* MADV_HUGEPAGE */);
   if (ret == -1) {
     PrintSystemError(errno);
     ret = munmap(tmem, size);

(MADV_HUGEPAGE was added in Linux 2.6.38 and our baseline for running node is 3.x but I don't know what exactly we require for building node.)

Moves the option that instructs Node.js to-remap its static code to
large pages from a configure-time option to a runtime option. This
should make it easy to assess the performance impact of such a change
without having to custom-build.
@gabrielschulhof
Copy link
Contributor Author

@addaleax I added the test and I fixed the doc.
@bnoordhuis I replaced the constant with its value and added the man page entry.

src/node.cc Outdated Show resolved Hide resolved
@gabrielschulhof
Copy link
Contributor Author

@bnoordhuis @addaleax @cjihrig @devnexen @gireeshpunathil I changed the implementation to one that accepts three values: 0 (default) to not map, 1 to map but to not report failure, and 2 to map and to report failure on stderr.

doc/node.1 Outdated Show resolved Hide resolved
Copy link
Member

@lundibundi lundibundi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think at least IsLargePagesEnabled and MapStaticCodeToLargePages in node_large_page.cc have to be changed as they won't compile/won't be correct on unsupported platforms due to conditional compilation in there. I think just adding conditional #else with correct error ret will be fine. They both won't have a return statement on the unsupported platforms.

src/node.cc Outdated Show resolved Hide resolved
test/parallel/test-startup-large-pages.js Outdated Show resolved Hide resolved
@nodejs-github-bot
Copy link
Collaborator

@gabrielschulhof
Copy link
Contributor Author

@addaleax @joyeecheung I updated the possible values to strings: "off", "silent", "verbose".
@lundibundi I applied your nits, and I added reporting for the case where mapping is not supported. I also placed the whole chunk into an ifdef to prevent compiling the code on unsupported platforms. On those platforms and in the verbose case, a message will be written to stderr that they're not supported.

@nodejs-github-bot
Copy link
Collaborator

@gabrielschulhof
Copy link
Contributor Author

@devnexen thanks, updated.

@nodejs-github-bot
Copy link
Collaborator

doc/api/cli.md Outdated Show resolved Hide resolved
src/node_options.cc Show resolved Hide resolved
@gabrielschulhof
Copy link
Contributor Author

Dang! I made a copy/paste error while landing. Sorry I omitted you as a reviewer @jasnell!

@bcoe
Copy link
Contributor

bcoe commented Dec 25, 2019

@gabrielschulhof we're seeing a bit of a hiccup in our nightly linux tests:

20:01:53 /bin/sh: 1: realpath: not found
20:01:53 gyp: Call to 'realpath src/large_pages/ld.implicit.script' returned exit status 127 while in /home/iojs/build/workspace/node-test-commit-linux-coverage-daily/nodes/benchmark/node.gyp.

Which seems to crop up somewhere between December 21st and December 22nd when this landed. I'm guessing there might be a workaround in changing our compiler flags? Wondering if this is expected however (would guess moving this to a flag was meant to be a noop?).

@bcoe bcoe mentioned this pull request Dec 26, 2019
4 tasks
gabrielschulhof pushed a commit to gabrielschulhof/node that referenced this pull request Dec 31, 2019
Moves the option that instructs Node.js to-remap its static code to
large pages from a configure-time option to a runtime option. This
should make it easy to assess the performance impact of such a change
without having to custom-build.

PR-URL: nodejs#30954
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: David Carlier <[email protected]>
Reviewed-By: David Carlier <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Co-authored-by: David Carlier <[email protected]>
BridgeAR pushed a commit that referenced this pull request Jan 3, 2020
Moves the option that instructs Node.js to-remap its static code to
large pages from a configure-time option to a runtime option. This
should make it easy to assess the performance impact of such a change
without having to custom-build.

PR-URL: #30954
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: David Carlier <[email protected]>
Reviewed-By: David Carlier <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Co-authored-by: David Carlier <[email protected]>
@BridgeAR BridgeAR mentioned this pull request Jan 7, 2020
MylesBorins pushed a commit that referenced this pull request Jan 8, 2020
Moves the option that instructs Node.js to-remap its static code to
large pages from a configure-time option to a runtime option. This
should make it easy to assess the performance impact of such a change
without having to custom-build.

Backport-PR-URL: #31063
PR-URL: #30954
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: David Carlier <[email protected]>
Reviewed-By: David Carlier <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Co-authored-by: David Carlier <[email protected]>
BethGriggs pushed a commit that referenced this pull request Feb 6, 2020
Moves the option that instructs Node.js to-remap its static code to
large pages from a configure-time option to a runtime option. This
should make it easy to assess the performance impact of such a change
without having to custom-build.

Backport-PR-URL: #31063
PR-URL: #30954
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: David Carlier <[email protected]>
Reviewed-By: David Carlier <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Co-authored-by: David Carlier <[email protected]>
@MylesBorins MylesBorins mentioned this pull request Feb 8, 2020
@gabrielschulhof gabrielschulhof deleted the runtime-large-pages-2 branch February 21, 2020 16:39
gabrielschulhof pushed a commit to gabrielschulhof/node that referenced this pull request Mar 6, 2020
Moves the option that instructs Node.js to-remap its static code to
large pages from a configure-time option to a runtime option. This
should make it easy to assess the performance impact of such a change
without having to custom-build.

PR-URL: nodejs#30954
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: David Carlier <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Co-authored-by: David Carlier <[email protected]>
gabrielschulhof pushed a commit to gabrielschulhof/node that referenced this pull request Apr 9, 2020
Moves the option that instructs Node.js to-remap its static code to
large pages from a configure-time option to a runtime option. This
should make it easy to assess the performance impact of such a change
without having to custom-build.

PR-URL: nodejs#30954
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: David Carlier <[email protected]>
Reviewed-By: David Carlier <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Co-authored-by: David Carlier <[email protected]>
targos pushed a commit to targos/node that referenced this pull request Apr 25, 2020
Moves the option that instructs Node.js to-remap its static code to
large pages from a configure-time option to a runtime option. This
should make it easy to assess the performance impact of such a change
without having to custom-build.

Backport-PR-URL: nodejs#32092
PR-URL: nodejs#30954
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: David Carlier <[email protected]>
Reviewed-By: David Carlier <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Co-authored-by: David Carlier <[email protected]>
targos pushed a commit that referenced this pull request Apr 28, 2020
Moves the option that instructs Node.js to-remap its static code to
large pages from a configure-time option to a runtime option. This
should make it easy to assess the performance impact of such a change
without having to custom-build.

Backport-PR-URL: #32092
PR-URL: #30954
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: David Carlier <[email protected]>
Reviewed-By: David Carlier <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Denys Otrishko <[email protected]>
Co-authored-by: David Carlier <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory.
Projects
None yet
Development

Successfully merging this pull request may close these issues.