Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v12.22.0 proposal #37797

Merged
merged 17 commits into from
Mar 30, 2021
Merged

v12.22.0 proposal #37797

merged 17 commits into from
Mar 30, 2021

Conversation

richardlau
Copy link
Member

@richardlau richardlau commented Mar 18, 2021

2021-03-30, Version 12.22.0 'Erbium' (LTS), @richardlau

Notable changes

The legacy HTTP parser is runtime deprecated

The legacy HTTP parser, selected by the --http-parser=legacy command line
option, is deprecated with the pending End-of-Life of Node.js 10.x (where it
is the only HTTP parser implementation provided) at the end of April 2021. It
will now warn on use but otherwise continue to function and may be removed in
a future Node.js 12.x release.

The default HTTP parser based on llhttp is not affected. By default it is
stricter than the now deprecated legacy HTTP parser. If interoperability with
HTTP implementations that send invalid HTTP headers is required, the HTTP
parser can be started in a less secure mode with the
--insecure-http-parser
command line option.

Contributed by Beth Griggs #37603.

ES Modules

ES Modules are now considered stable.

Contributed by Guy Bedford #35781

node-api

Updated to node-api version 8 and added an experimental API to allow
retrieval of the add-on file name.

Contributed by Gabriel Schulhof #37652 and #37195.

New API's to control code coverage data collection

v8.stopCoverage() and v8.takeCoverage() have been added.

Contributed by Joyee Cheung #33807.

New API to monitor event loop utilization by Worker threads

worker.performance.eventLoopUtilization() has been added.

Contributed by Trevor Norris #35664.

Commits

  • [1872625990] - (SEMVER-MINOR) deps: update to [email protected] (Guy Bedford) #37712
  • [dfa04d9035] - deps: V8: cherry-pick beebee4f80ff (Peter Marshall) #37293
  • [bf8733fe22] - doc: mark modules implementation as stable (Guy Bedford) #35781
  • [0a35d49f56] - Revert "embedding: make Stop() stop Workers" (Anna Henningsen) #32623
  • [a0b610450a] - (SEMVER-MINOR) http: runtime deprecate legacy HTTP parser (Beth Griggs) #37603
  • [2da24ac302] - lib: add URI handling functions to primordials (Antoine du Hamel) #37394
  • [7b0ed4ba92] - module: improve support of data: URLs (Antoine du Hamel) #37392
  • [93dd799a86] - (SEMVER-MINOR) node-api: define version 8 (Gabriel Schulhof) #37652
  • [f5692093d3] - (SEMVER-MINOR) node-api: allow retrieval of add-on file name (Gabriel Schulhof) #37195
  • [6cef0e3678] - src,test: add regression test for nested Worker termination (Anna Henningsen) #32623
  • [364bf03a68] - test: fix races in test-performance-eventlooputil (Gerhard Stoebich) #36028
  • [d7a4ccdf09] - test: correct test-worker-eventlooputil (Gerhard Stoebich) #35891
  • [0f6d44500c] - test: add cpu-profiler-crash test (Santiago Gimeno) #37293
  • [86f34ee18c] - (SEMVER-MINOR) v8: implement v8.stopCoverage() (Joyee Cheung) #33807
  • [8ddea3f16d] - (SEMVER-MINOR) v8: implement v8.takeCoverage() (Joyee Cheung) #33807
  • [eec7542781] - (SEMVER-MINOR) worker: add eventLoopUtilization() (Trevor Norris) #35664

cc @nodejs/releasers

addaleax and others added 12 commits February 23, 2021 11:24
This reverts commit 037ac99.

As flaky CI failures have revealed, this feature was implemented
incorrectly. `stop_sub_worker_contexts()` needs to be called on the
thread on which the `Environment` is currently running, it’s not
thread-safe. The current API requires `Stop()` to be thread-safe,
though.

We could add a new API for this, but unless there’s demand, that’s
probably not necessary as `FreeEnvironment()` will also stop Workers,
which is commonly the next action on an `Environment` instance after
having `Stop()` called on it.

Refs: #32531

PR-URL: #32623
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
This adds a regression test for terminating a Worker inside which
another Worker is running.

PR-URL: #32623
Refs: #32531
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
The legacy HTTP parser, used by default in versions of Node.js prior to
12.0.0, is deprecated. The legacy HTTP parser cannot be guaranteed to be
supported after April 2021. This commit introduces a deprecation warning
for the legacy HTTP parser.

PR-URL: #37603
Refs: #31441
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Original commit message:
```
cpu-profiler: Use Handle version of SourcePositionTableIterator

The surrounding code can trigger an allocation through InliningStack
which can eventually end up allocating a line ends array.

This is fine as-is because the existing iterator code makes a copy
of the byte array. It just triggers the no_gc dcheck in debug mode.

Fixed: v8:10778
Change-Id: Ic8c502767ec6c3d3b1f5e84df60638bd2fc6be75
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2339102
Auto-Submit: Peter Marshall <[email protected]>
Commit-Queue: Tobias Tebbi <[email protected]>
Reviewed-by: Tobias Tebbi <[email protected]>
Cr-Commit-Position: refs/heads/master@{#69247}
```

Refs: v8/v8@beebee4

PR-URL: #37293
Backport-PR-URL: #37578
Reviewed-By: Jiawen Geng <[email protected]>
Reviewed-By: Juan José Arboleda <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
This test is added as it usually crashes without applying the v8 patch:
v8/v8@beebee4

PR-URL: #37293
Backport-PR-URL: #37578
Refs: v8/v8@beebee4
Reviewed-By: Jiawen Geng <[email protected]>
Reviewed-By: Juan José Arboleda <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Allow calling eventLoopUtilization() directly on a worker thread:

    const worker = new Worker('./foo.js');
    const elu = worker.performance.eventLoopUtilization();
    setTimeout(() => {
      worker.performance.eventLoopUtilization(elu);
    }, 10);

Add a new performance object on the Worker instance that will hopefully
one day hold all the other performance metrics, such as nodeTiming.

Include benchmarks and tests.

PR-URL: #35664
Reviewed-By: Juan José Arboleda <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Gerhard Stöbich <[email protected]>
Reviewed-By: James M Snell <[email protected]>

Backport-PR-URL: #37165
The active worker check compared the time from sending message till
response arrived from worker with the complete time the worker was
running till it responses to the spin request.

If sending back the message is slow for some reason the test fails.

Adapt the test to compare the time seen inside the worker with the
time read from main thread.

PR-URL: #35891
Fixes: #35844
Refs: #35886
Refs: #35664
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: Rich Trott <[email protected]>

Backport-PR-URL: #37165
Add an v8.takeCoverage() API that allows the user to write the
coverage started by NODE_V8_COVERAGE to disk on demand.
The coverage can be written multiple times during the lifetime
of the process, each time the execution counter will be reset.
When the process is about to exit, one last coverage will
still be written to disk.

Also refactors the internal profiler connection code
so that we use the inspector response id to identify
the profile response instead of using an ad-hoc flag in C++.

PR-URL: #33807
Backport-PR-URL: #36352
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Jiawen Geng <[email protected]>
Reviewed-By: Ben Coe <[email protected]>
Add a v8.stopCoverage() API to stop the coverage collection
started by NODE_V8_COVERAGE - this would be useful in
conjunction with v8.takeCoverage() if the user don't want
to emit the coverage at the process exit but still want
to collect it on demand at some point.

PR-URL: #33807
Backport-PR-URL: #36352
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Jiawen Geng <[email protected]>
Reviewed-By: Ben Coe <[email protected]>
Unlike JS-only modules, native add-ons are always associated with a
dynamic shared object from which they are loaded. Being able to
retrieve its absolute path is important to native-only add-ons, i.e.
add-ons that are not themselves being loaded from a JS-only module
located in the same package as the native add-on itself.

Currently, the file name is obtained at environment construction time
from the JS `module.filename`. Nevertheless, the presence of `module`
is not required, because the file name could also be passed in via a
private property added onto `exports` from the `process.dlopen`
binding.

As an attempt at future-proofing, the file name is provided as a URL,
i.e. prefixed with the `file://` protocol.

Fixes: nodejs/node-addon-api#449
PR-URL: #37195
Backport-PR-URL: #37328
Co-authored-by: Michael Dawson <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
PR-URL: #35781
Backport-PR-URL: #37718
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Franziska Hinkelmann <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Myles Borins <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Derek Lewis <[email protected]>
PR-URL: #37712
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Myles Borins <[email protected]>
Reviewed-By: Bradley Farias <[email protected]>
Reviewed-By: Geoffrey Booth <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
@nodejs-github-bot nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. v12.x labels Mar 18, 2021
richardlau added a commit that referenced this pull request Mar 18, 2021
Notable changes

The legacy HTTP parser is runtime deprecated:
- The legacy HTTP parser, selected by the `--http-parser=legacy` command line
option, is deprecated with the pending End-of-Life of Node.js 10.x (where it
is the only HTTP parser implementation provided) at the end of April 2021. It
will now warn on use but otherwise continue to function and may be removed in
a future Node.js 12.x release.
- The default HTTP parser based on llhttp is not affected. By default it is
stricter than the now deprecated legacy HTTP parser. If interoperability with
HTTP implementations that send invalid HTTP headers is required, the HTTP
parser can be started in a less secure mode with the `--insecure-http-parser`
command line option.

ES Modules:
- ES Modules are now considered stable.

node-api:
- Added an experimental API to allow retrieval of the add-on file name.

New API's to control code coverage data collection:
- `v8.stopCoverage()` and `v8.takeCoverage()` have been added.

New API to monitor event loop utilization by Worker threads
- `worker.performance.eventLoopUtilization()` has been added.

PR-URL: #37797
@richardlau
Copy link
Member Author

@nodejs/releasers I'm aiming to get this out on the 30 March 2021. That gives us a month before Node.js 10.x goes End-of-Life (see the notable changes about the legacy HTTP parser deprecation).

Since this is a semver-minor I've pulled in #37712 which has just gone out in Node.js 15.12.0 but will have been out for just less than two weeks by 30 March 2021.

I am planning on also including #37796 (need to wait for CI's to complete/pass).

Any suggestions for improvements to the notable changes section welcome.

Mark as stable the APIs that define Node-API version 8.

PR-URL: #37652
Backport-PR-URL: #37796
Reviewed-By: Gerhard Stöbich <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
richardlau added a commit that referenced this pull request Mar 19, 2021
Notable changes

The legacy HTTP parser is runtime deprecated:
- The legacy HTTP parser, selected by the `--http-parser=legacy` command line
option, is deprecated with the pending End-of-Life of Node.js 10.x (where it
is the only HTTP parser implementation provided) at the end of April 2021. It
will now warn on use but otherwise continue to function and may be removed in
a future Node.js 12.x release.
- The default HTTP parser based on llhttp is not affected. By default it is
stricter than the now deprecated legacy HTTP parser. If interoperability with
HTTP implementations that send invalid HTTP headers is required, the HTTP
parser can be started in a less secure mode with the `--insecure-http-parser`
command line option.

ES Modules:
- ES Modules are now considered stable.

node-api:
- Updated to node-api version 8 and added an experimental API to allow
retrieval of the add-on file name.

New API's to control code coverage data collection:
- `v8.stopCoverage()` and `v8.takeCoverage()` have been added.

New API to monitor event loop utilization by Worker threads
- `worker.performance.eventLoopUtilization()` has been added.

PR-URL: #37797
@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Mar 19, 2021

@richardlau
Copy link
Member Author

undici is failing in CITGM with the current Node.js 12.22.0 proposal across multiple (probably all as not all have completed) platforms:

e.g. https://ci.nodejs.org/job/citgm-smoker/2649/nodes=rhel7-s390x/testReport/junit/(root)/citgm/undici_v3_3_3/

 not ok 13 - test/client-pipelining.js # time=499.442ms
   ---
   env: {}
   file: test/client-pipelining.js
   timeout: 60000
   command: /data/iojs/build/workspace/citgm-smoker/nodes/rhel7-s390x/smoker/bin/node
   args:
     - --expose-gc
     - test/client-pipelining.js
   stdio:
     - 0
     - pipe
     - 2
   cwd: /data/iojs/tmp/citgm_tmp/0fb9c5c5-d819-45c6-a028-9ff49a2184da/undici
   exitCode: 1
   ...
 {
     # Subtest: 20 times GET with pipelining 10
         1..61
         not ok 1 - (unnamed test)
           ---
           code: DEP0131
           ...
         
...
 not ok 18 - test/client-write-max-listeners.js # time=55.518ms
   ---
   env: {}
   file: test/client-write-max-listeners.js
   timeout: 60000
   command: /data/iojs/build/workspace/citgm-smoker/nodes/rhel7-s390x/smoker/bin/node
   args:
     - --expose-gc
     - test/client-write-max-listeners.js
   stdio:
     - 0
     - pipe
     - 2
   cwd: /data/iojs/tmp/citgm_tmp/0fb9c5c5-d819-45c6-a028-9ff49a2184da/undici
   exitCode: 1
   ...
 {
     # Subtest: socket close listener does not leak
         1..32
         not ok 1 - (unnamed test)
           ---
           at:
             line: 34
             column: 7
             file: test/client-write-max-listeners.js
             type: process
           stack: |
             process.<anonymous> (test/client-write-max-listeners.js:34:7)
             process.emit (node_modules/source-map-support/source-map-support.js:495:21)
             process.domainProcessEmit (node_modules/async-hook-domain/index.js:137:26)
           source: |2
               process.on('warning', () => {
                 t.fail()
             ------^
               })
           ...

I can reproduce locally on Fedora 33. It looks like the new warning for the legacy HTTP parser deprecation is being emitted and causing two of the tests to fail. I believe nodejs/undici#564 will fix it as running citgm against the current main branch of undici (node bin/citgm nodejs/undici#main) passes, while node bin/citgm undici and node bin/citgm nodejs/undici#3.x fails.

FWIW process.binding('http_parser'), which is what undici 3.3.3 does, will load the legacy http parser in Node.js 12:

node/lib/_http_common.js

Lines 33 to 34 in 1b4790b

getOptionValue('--http-parser') === 'legacy' ?
internalBinding('http_parser') : internalBinding('http_parser_llhttp');

FYI @nodejs/undici

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@aduh95
Copy link
Contributor

aduh95 commented Mar 21, 2021

I think it would make sense to include #37392 given #35781.

@richardlau
Copy link
Member Author

I think it would make sense to include #37392 given #35781.

@aduh95 It does not cherry-pick cleanly to v12.x-staging.

@richardlau
Copy link
Member Author

richardlau commented Mar 22, 2021

Underscore CITGM failure:

e.g. https://ci.nodejs.org/job/citgm-smoker/2649/nodes=fedora-last-latest-x64/testReport/junit/(root)/citgm/underscore_v1_12_1/

https://github.com/jashkenas/underscore/issues/2915#issuecomment-802789289Error: Failure getting project from npm
npm ERR! code E404
npm ERR! 404 Not Found - GET https://codeload.github.com/jashkenas/underscore/tar.gz/bf5a0ed27599f99ea59a0839c5bc2fb27a46c1cf
npm ERR! 404 
npm ERR! 404  'https://github.com/jashkenas/underscore/archive/bf5a0ed27599f99ea59a0839c5bc2fb27a46c1cf.tar.gz' is not in the npm registry.
npm ERR! 404 Your package name is not valid, because 
npm ERR! 404  1. name can only contain URL-friendly characters
npm ERR! 404 
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/iojs/tmp/citgm_tmp/a7df1de0-6c8a-4cd9-8f20-b033c89b83da/home/.npm/_logs/2021-03-19T15_05_00_202Z-debug.log

looks to be because there's no tag (deliberately at this time) for Underscore 1.12.1 in GitHub: jashkenas/underscore#2915 (comment)

PR-URL: #37394
Backport-PR-URL: #37859
Reviewed-By: Zijian Liu <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Add support for loading modules using percent-encoded URLs.

PR-URL: #37392
Backport-PR-URL: #37859
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Bradley Farias <[email protected]>
Reviewed-By: Myles Borins <[email protected]>
Reviewed-By: Jan Krems <[email protected]>
Reviewed-By: Guy Bedford <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Zijian Liu <[email protected]>
richardlau added a commit that referenced this pull request Mar 29, 2021
Notable changes

The legacy HTTP parser is runtime deprecated:
- The legacy HTTP parser, selected by the `--http-parser=legacy` command line
option, is deprecated with the pending End-of-Life of Node.js 10.x (where it
is the only HTTP parser implementation provided) at the end of April 2021. It
will now warn on use but otherwise continue to function and may be removed in
a future Node.js 12.x release.
- The default HTTP parser based on llhttp is not affected. By default it is
stricter than the now deprecated legacy HTTP parser. If interoperability with
HTTP implementations that send invalid HTTP headers is required, the HTTP
parser can be started in a less secure mode with the `--insecure-http-parser`
command line option.

ES Modules:
- ES Modules are now considered stable.

node-api:
- Updated to node-api version 8 and added an experimental API to allow
retrieval of the add-on file name.

New API's to control code coverage data collection:
- `v8.stopCoverage()` and `v8.takeCoverage()` have been added.

New API to monitor event loop utilization by Worker threads
- `worker.performance.eventLoopUtilization()` has been added.

PR-URL: #37797
@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Mar 29, 2021

Flarna and others added 2 commits March 29, 2021 14:10
Fix two races in test-performance-eventlooputil resulting in a flaky
test.

elu1 was capture after start time t from spin look. If OS descides to
reschedule the process after capturing t but before getting elu for
>=50ms the spin loop is actually a nop. elu1 doesn't show this and as
a result elut3 = eventLoopUtilization(elu1) results in
elu3.active === 0.
Moving capturing of t after capturing t, just before the spin look
avoids this.

Similar if OS decides to shedule a different process between getting
the total elu from start and the diff elu showing the spin loop the
check to verify that total active time is long then the spin loop
fails.
Exchanging these statements avoids this race.

PR-URL: #36028
Fixes: #35309
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Notable changes

The legacy HTTP parser is runtime deprecated:
- The legacy HTTP parser, selected by the `--http-parser=legacy` command line
option, is deprecated with the pending End-of-Life of Node.js 10.x (where it
is the only HTTP parser implementation provided) at the end of April 2021. It
will now warn on use but otherwise continue to function and may be removed in
a future Node.js 12.x release.
- The default HTTP parser based on llhttp is not affected. By default it is
stricter than the now deprecated legacy HTTP parser. If interoperability with
HTTP implementations that send invalid HTTP headers is required, the HTTP
parser can be started in a less secure mode with the `--insecure-http-parser`
command line option.

ES Modules:
- ES Modules are now considered stable.

node-api:
- Updated to node-api version 8 and added an experimental API to allow
retrieval of the add-on file name.

New API's to control code coverage data collection:
- `v8.stopCoverage()` and `v8.takeCoverage()` have been added.

New API to monitor event loop utilization by Worker threads
- `worker.performance.eventLoopUtilization()` has been added.

PR-URL: #37797
@nodejs-github-bot
Copy link
Collaborator

@richardlau
Copy link
Member Author

Saw some parallel/test-performance-eventlooputil failures in the CI on macOS. I've cherry-picked #36028, which fixes some race conditions in the test.

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Mar 29, 2021

@richardlau
Copy link
Member Author

@richardlau richardlau merged commit 6a5797b into v12.x Mar 30, 2021
richardlau added a commit that referenced this pull request Mar 30, 2021
richardlau added a commit that referenced this pull request Mar 30, 2021
Notable changes

The legacy HTTP parser is runtime deprecated:
- The legacy HTTP parser, selected by the `--http-parser=legacy` command line
option, is deprecated with the pending End-of-Life of Node.js 10.x (where it
is the only HTTP parser implementation provided) at the end of April 2021. It
will now warn on use but otherwise continue to function and may be removed in
a future Node.js 12.x release.
- The default HTTP parser based on llhttp is not affected. By default it is
stricter than the now deprecated legacy HTTP parser. If interoperability with
HTTP implementations that send invalid HTTP headers is required, the HTTP
parser can be started in a less secure mode with the `--insecure-http-parser`
command line option.

ES Modules:
- ES Modules are now considered stable.

node-api:
- Updated to node-api version 8 and added an experimental API to allow
retrieval of the add-on file name.

New API's to control code coverage data collection:
- `v8.stopCoverage()` and `v8.takeCoverage()` have been added.

New API to monitor event loop utilization by Worker threads
- `worker.performance.eventLoopUtilization()` has been added.

PR-URL: #37797
richardlau added a commit to richardlau/nodejs.org that referenced this pull request Mar 30, 2021
richardlau added a commit to nodejs/nodejs.org that referenced this pull request Mar 30, 2021
@richardlau richardlau deleted the v12.22.0-proposal branch March 30, 2021 14:27
@targos targos added the release Issues and PRs related to Node.js releases. label Apr 11, 2021
@targos targos removed lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Jun 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release Issues and PRs related to Node.js releases.
Projects
None yet
Development

Successfully merging this pull request may close these issues.