Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

src: enable detailed source positions in V8 #24274

Closed
wants to merge 2 commits into from

Conversation

hashseed
Copy link
Member

@hashseed hashseed commented Nov 9, 2018

Enable detailed source positions in generated code. When profiling, this gives a better mapping between profile samples to source text.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • commit message follows commit guidelines

Original commit message:

  [profiler] introduce API to enable detailed source positions

  This allows Node.js to enable detailed source positions for optimized code
  early on, without having to pass a flag string.

  [email protected]

  Change-Id: Ie74ea41f600cf6e31acbe802116df4976ccf1c75
  Reviewed-on: https://chromium-review.googlesource.com/c/1319757
  Commit-Queue: Yang Guo <[email protected]>
  Reviewed-by: Peter Marshall <[email protected]>
  Cr-Commit-Position: refs/heads/master@{#57380}
@nodejs-github-bot nodejs-github-bot added the lib / src Issues and PRs related to general changes in the lib or src directory. label Nov 9, 2018
@psmarshall
Copy link
Contributor

@psmarshall psmarshall added v8 engine Issues and PRs related to the V8 dependency. author ready PRs that have at least one approval, no pending requests for changes, and a CI started. labels Nov 9, 2018
@@ -2548,6 +2548,7 @@ Isolate* NewIsolate(ArrayBufferAllocator* allocator, uv_loop_t* event_loop) {
isolate->SetMicrotasksPolicy(MicrotasksPolicy::kExplicit);
isolate->SetFatalErrorHandler(OnFatalError);
isolate->SetAllowWasmCodeGenerationCallback(AllowWasmCodeGenerationCallback);
v8::CpuProfiler::UseDetailedSourcePositionsForProfiling(isolate);
Copy link
Contributor

@refack refack Nov 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: does this have a penalty? And even if not could this be conditional on inspector being attached or profiler run requested (I mean is it possible)?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with only enabling this once the profiler run is requested is that functions that get optimized before profiling starts don't have the detailed line information. In the browser you can work around this by reloading the page with profiling enabled (devtools has a button for this) but for a running Node instance this is not really feasible, particularly if you want to debug some rarely-occurring thing. Unfortunately we see this in the wild quite a bit so having it on by default makes sense IMO.

The overhead is very low - on average it makes no difference, it the worst case a few percent increase in optimized compile time, but this is currently being moved off-thread anyway (plus the general performance boost of upgrading the V8 version is going to easily outweigh this).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation.

Copy link
Contributor

@refack refack Nov 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the browser you can work around this by reloading the page with profiling enabled

I'm thinking about this some more...
For node we have one main way to enable inspetor, via CLI args. There are two other ways which I don't have usage information about SIGUSR and inspector.open().

It might be a better balance to turn this on only for processes started with inspector listening, and save the performance cost for regular runs. IIUC the down side is less accurate code position information?

I'm ambivalent...

/CC @nodejs/performance

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CpuProfiler can be also used without inspector, see e.g. NPM modules like v8-profiler. There are tools out in the wild which start profiling automatically for some time if they notice performance issues.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's also possible to use the CPU profiler directly through the V8 API without the inspector and there are some native modules that do this - that approach wouldn't work for them as far as I understand

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thoughts leading to this were:

  • It's hard to foresee whether the user is going to profile, both for Chrome and Node.
  • For Chrome, you can at least reload, with DevTools open.
  • Node.js does not have the reload option.
  • Adding more accurate data has a small one-time cost for the script. Chrome suffers a bit more for this as you pay more for optimizing compiles, possibly while starting up a page, and this occurs on every page navigation. Node.js does not encounter new scripts to compile and optimize as often.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that it's a tough call:
slightly slower compile time but for everyone ⚖️ improvement not available but only for profiling started out-of-band

@hashseed
Copy link
Member Author

Seems like we got a few timeouts on SmartOS.

How should we proceed with this PR?

@refack
Copy link
Contributor

refack commented Nov 13, 2018

They might be flakes https://ci.nodejs.org/job/node-test-pull-request/18574/

@hashseed
Copy link
Member Author

Looks like some other tests failed this time.

@Trott
Copy link
Member

Trott commented Nov 14, 2018

Trott pushed a commit to Trott/io.js that referenced this pull request Nov 14, 2018
Original commit message:

  [profiler] introduce API to enable detailed source positions

  This allows Node.js to enable detailed source positions for optimized code
  early on, without having to pass a flag string.

  [email protected]

  Change-Id: Ie74ea41f600cf6e31acbe802116df4976ccf1c75
  Reviewed-on: https://chromium-review.googlesource.com/c/1319757
  Commit-Queue: Yang Guo <[email protected]>
  Reviewed-by: Peter Marshall <[email protected]>
  Cr-Commit-Position: refs/heads/master@{#57380}

PR-URL: nodejs#24274
Reviewed-By: Gus Caplan <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: Ali Ijaz Sheikh <[email protected]>
Reviewed-By: Matheus Marchini <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Trott pushed a commit to Trott/io.js that referenced this pull request Nov 14, 2018
PR-URL: nodejs#24274
Reviewed-By: Gus Caplan <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: Ali Ijaz Sheikh <[email protected]>
Reviewed-By: Matheus Marchini <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
@Trott
Copy link
Member

Trott commented Nov 14, 2018

Landed in c1d18a0...e2a8e32

@Trott Trott closed this Nov 14, 2018
BridgeAR pushed a commit that referenced this pull request Nov 14, 2018
Original commit message:

  [profiler] introduce API to enable detailed source positions

  This allows Node.js to enable detailed source positions for optimized code
  early on, without having to pass a flag string.

  [email protected]

  Change-Id: Ie74ea41f600cf6e31acbe802116df4976ccf1c75
  Reviewed-on: https://chromium-review.googlesource.com/c/1319757
  Commit-Queue: Yang Guo <[email protected]>
  Reviewed-by: Peter Marshall <[email protected]>
  Cr-Commit-Position: refs/heads/master@{#57380}

PR-URL: #24274
Reviewed-By: Gus Caplan <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: Ali Ijaz Sheikh <[email protected]>
Reviewed-By: Matheus Marchini <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
BridgeAR pushed a commit that referenced this pull request Nov 14, 2018
PR-URL: #24274
Reviewed-By: Gus Caplan <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: Ali Ijaz Sheikh <[email protected]>
Reviewed-By: Matheus Marchini <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
kiyomizumia pushed a commit to kiyomizumia/node that referenced this pull request Nov 15, 2018
Original commit message:

  [profiler] introduce API to enable detailed source positions

  This allows Node.js to enable detailed source positions for optimized code
  early on, without having to pass a flag string.

  [email protected]

  Change-Id: Ie74ea41f600cf6e31acbe802116df4976ccf1c75
  Reviewed-on: https://chromium-review.googlesource.com/c/1319757
  Commit-Queue: Yang Guo <[email protected]>
  Reviewed-by: Peter Marshall <[email protected]>
  Cr-Commit-Position: refs/heads/master@{#57380}

PR-URL: nodejs#24274
Reviewed-By: Gus Caplan <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: Ali Ijaz Sheikh <[email protected]>
Reviewed-By: Matheus Marchini <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
kiyomizumia pushed a commit to kiyomizumia/node that referenced this pull request Nov 15, 2018
PR-URL: nodejs#24274
Reviewed-By: Gus Caplan <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: Ali Ijaz Sheikh <[email protected]>
Reviewed-By: Matheus Marchini <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
BridgeAR pushed a commit that referenced this pull request Nov 15, 2018
Original commit message:

  [profiler] introduce API to enable detailed source positions

  This allows Node.js to enable detailed source positions for optimized code
  early on, without having to pass a flag string.

  [email protected]

  Change-Id: Ie74ea41f600cf6e31acbe802116df4976ccf1c75
  Reviewed-on: https://chromium-review.googlesource.com/c/1319757
  Commit-Queue: Yang Guo <[email protected]>
  Reviewed-by: Peter Marshall <[email protected]>
  Cr-Commit-Position: refs/heads/master@{#57380}

PR-URL: #24274
Reviewed-By: Gus Caplan <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: Ali Ijaz Sheikh <[email protected]>
Reviewed-By: Matheus Marchini <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
BridgeAR pushed a commit that referenced this pull request Nov 15, 2018
PR-URL: #24274
Reviewed-By: Gus Caplan <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: Ali Ijaz Sheikh <[email protected]>
Reviewed-By: Matheus Marchini <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
@targos targos mentioned this pull request Nov 16, 2018
@targos
Copy link
Member

targos commented Nov 17, 2018

The commits from this PR were reverted in #24394 (V8 tests were broken). Please open a new pull request to get them back in.

N.B.: I didn't add the dont-land-on-v11.x label because this was already released in v11.2.0.

@hashseed hashseed deleted the detailedsourcepositions branch November 20, 2018 08:15
addaleax pushed a commit that referenced this pull request Nov 23, 2018
Original commit message:

    [profiler] introduce API to enable detailed source positions

    This allows Node.js to enable detailed source positions for optimized code
    early on, without having to pass a flag string.

    [email protected]

    Change-Id: Ie74ea41f600cf6e31acbe802116df4976ccf1c75
    Reviewed-on: https://chromium-review.googlesource.com/c/1319757
    Commit-Queue: Yang Guo <[email protected]>
    Reviewed-by: Peter Marshall <[email protected]>
    Cr-Commit-Position: refs/heads/master@{#57380}

Refs: v8/v8@073073b

PR-URL: #24515
Refs: #24274
Refs: #24394
Refs: #24393
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
addaleax pushed a commit that referenced this pull request Nov 23, 2018
PR-URL: #24515
Refs: #24274
Refs: #24394
Refs: #24393
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
targos pushed a commit that referenced this pull request Nov 24, 2018
Original commit message:

    [profiler] introduce API to enable detailed source positions

    This allows Node.js to enable detailed source positions for optimized code
    early on, without having to pass a flag string.

    [email protected]

    Change-Id: Ie74ea41f600cf6e31acbe802116df4976ccf1c75
    Reviewed-on: https://chromium-review.googlesource.com/c/1319757
    Commit-Queue: Yang Guo <[email protected]>
    Reviewed-by: Peter Marshall <[email protected]>
    Cr-Commit-Position: refs/heads/master@{#57380}

Refs: v8/v8@073073b

PR-URL: #24515
Refs: #24274
Refs: #24394
Refs: #24393
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
targos pushed a commit that referenced this pull request Nov 24, 2018
PR-URL: #24515
Refs: #24274
Refs: #24394
Refs: #24393
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
rvagg pushed a commit that referenced this pull request Nov 28, 2018
Original commit message:

    [profiler] introduce API to enable detailed source positions

    This allows Node.js to enable detailed source positions for optimized code
    early on, without having to pass a flag string.

    [email protected]

    Change-Id: Ie74ea41f600cf6e31acbe802116df4976ccf1c75
    Reviewed-on: https://chromium-review.googlesource.com/c/1319757
    Commit-Queue: Yang Guo <[email protected]>
    Reviewed-by: Peter Marshall <[email protected]>
    Cr-Commit-Position: refs/heads/master@{#57380}

Refs: v8/v8@073073b

PR-URL: #24515
Refs: #24274
Refs: #24394
Refs: #24393
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
rvagg pushed a commit that referenced this pull request Nov 28, 2018
PR-URL: #24515
Refs: #24274
Refs: #24394
Refs: #24393
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
targos pushed a commit to targos/node that referenced this pull request Dec 4, 2018
Original commit message:

    [profiler] introduce API to enable detailed source positions

    This allows Node.js to enable detailed source positions for optimized code
    early on, without having to pass a flag string.

    [email protected]

    Change-Id: Ie74ea41f600cf6e31acbe802116df4976ccf1c75
    Reviewed-on: https://chromium-review.googlesource.com/c/1319757
    Commit-Queue: Yang Guo <[email protected]>
    Reviewed-by: Peter Marshall <[email protected]>
    Cr-Commit-Position: refs/heads/master@{#57380}

Refs: v8/v8@073073b

PR-URL: nodejs#24515
Refs: nodejs#24274
Refs: nodejs#24394
Refs: nodejs#24393
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
pull bot pushed a commit to shakir-abdo/node that referenced this pull request Dec 6, 2018
Original commit message:

    [profiler] introduce API to enable detailed source positions

    This allows Node.js to enable detailed source positions for optimized code
    early on, without having to pass a flag string.

    [email protected]

    Change-Id: Ie74ea41f600cf6e31acbe802116df4976ccf1c75
    Reviewed-on: https://chromium-review.googlesource.com/c/1319757
    Commit-Queue: Yang Guo <[email protected]>
    Reviewed-by: Peter Marshall <[email protected]>
    Cr-Commit-Position: refs/heads/master@{#57380}

Refs: v8/v8@073073b

PR-URL: nodejs#24515
Refs: nodejs#24274
Refs: nodejs#24394
Refs: nodejs#24393
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
refack pushed a commit to refack/node that referenced this pull request Jan 14, 2019
Original commit message:

    [profiler] introduce API to enable detailed source positions

    This allows Node.js to enable detailed source positions for optimized code
    early on, without having to pass a flag string.

    [email protected]

    Change-Id: Ie74ea41f600cf6e31acbe802116df4976ccf1c75
    Reviewed-on: https://chromium-review.googlesource.com/c/1319757
    Commit-Queue: Yang Guo <[email protected]>
    Reviewed-by: Peter Marshall <[email protected]>
    Cr-Commit-Position: refs/heads/master@{#57380}

Refs: v8/v8@073073b

PR-URL: nodejs#24515
Refs: nodejs#24274
Refs: nodejs#24394
Refs: nodejs#24393
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
refack pushed a commit to refack/node that referenced this pull request Jan 14, 2019
PR-URL: nodejs#24515
Refs: nodejs#24274
Refs: nodejs#24394
Refs: nodejs#24393
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
refack pushed a commit to refack/node that referenced this pull request Jan 14, 2019
Original commit message:

    [profiler] introduce API to enable detailed source positions

    This allows Node.js to enable detailed source positions for optimized code
    early on, without having to pass a flag string.

    [email protected]

    Change-Id: Ie74ea41f600cf6e31acbe802116df4976ccf1c75
    Reviewed-on: https://chromium-review.googlesource.com/c/1319757
    Commit-Queue: Yang Guo <[email protected]>
    Reviewed-by: Peter Marshall <[email protected]>
    Cr-Commit-Position: refs/heads/master@{#57380}

Refs: v8/v8@073073b

PR-URL: nodejs#24515
Refs: nodejs#24274
Refs: nodejs#24394
Refs: nodejs#24393
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Peter Marshall <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Refael Ackermann <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. lib / src Issues and PRs related to general changes in the lib or src directory. v8 engine Issues and PRs related to the V8 dependency.
Projects
None yet
Development

Successfully merging this pull request may close these issues.