http2: backport changes to v9.x-staging #18050

jasnell · 2018-01-09T01:01:55Z

Backport of multiple http2 related commits to v9.x-staging.

Currently blocked by #18049 and whatever else is currently causing v9.x-staging to fail. Will be able to finish once v9.x-staging is functioning again.

Note: have not fully tested this yet because of the v9.x-staging breakage.

Ping @nodejs/http2 @mcollina

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)

http2

targos · 2018-01-09T07:22:59Z

v9.x-staging fails because #17704 is not in it. Is it possible to include it in this PR?

jasnell · 2018-01-09T07:26:50Z

Not sure. That's a semver-major pr so would need a semver-major minor backport that omitted the breaking bits. That would best be done in a separate PR.

mcollina · 2018-01-09T07:50:32Z

Could this commit be further backported to 8 in case?

jasnell · 2018-01-09T07:51:38Z

That's going to require a bit more work. Planning to do it later this week

MylesBorins · 2018-01-09T08:24:34Z

I've rebased against the new v9.x-staging to remove a commit that was breaking the build, it looks like another commit may be breaking stuff so i'll likely rebase one more time 😄

targos · 2018-01-09T08:34:12Z

@MylesBorins you are looking for #17800 which depends on #17704

MylesBorins · 2018-01-09T08:36:21Z

@targos you are a saint! I was just digging in

Adds the possibility to keep a strong persistent reference to a JS object while a `SetImmediate()` call is in effect. PR-URL: nodejs#17183 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Franziska Hinkelmann <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]>

Calling into JS land from GC is not allowed, so delay the resolution of pending pings when a session is destroyed. PR-URL: nodejs#17183 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Franziska Hinkelmann <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]>

Introduce an `Http2Scope` class that, when it goes out of scope, checks whether a write to the network is desired by nghttp2. If that is the case, schedule a write using `SetImmediate()` rather than a custom per-session libuv handle. PR-URL: nodejs#17183 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Franziska Hinkelmann <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]>

PR-URL: nodejs#17406 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]>

PR-URL: nodejs#17406 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]> This is a significant cleanup and refactoring of the cleanup/close/destroy logic for Http2Stream and Http2Session. There are significant changes here in the timing and ordering of cleanup logic, JS apis. and various related necessary edits.

`nghttp2_stream_write_t` was not a necessary redirection layer and came with the cost of one additional allocation per stream write. Also, having both `nghttp2_stream_write` and `nghttp2_stream_write_t` as identifiers did not help with readability. PR-URL: nodejs#17718 Reviewed-By: James M Snell <[email protected]>

- Only finish outgoing `WriteWrap`s once data has actually been passed to the underlying socket. - This makes HTTP2 streams respect backpressure - Use `DoTryWrite` as a shortcut for sending out as much of the data synchronously without blocking as possible - Use `NGHTTP2_DATA_FLAG_NO_COPY` to avoid copying DATA frame contents into nghttp2’s buffers before sending them out. PR-URL: nodejs#17718 Reviewed-By: James M Snell <[email protected]>

PR-URL: nodejs#17763 Refs: nodejs#17746 Reviewed-By: Matteo Collina <[email protected]>

PR-URL: nodejs#17863 Fixes: nodejs#17840 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>

Keep a local handle as a reference to the JS `Http2Session` object so that it will not be garbage collected when inside an `Http2Scope`, because the presence of the latter usually indicates that further actions on the session object are expected. Strictly speaking, storing the `session_handle_` as a property on the scope object is not necessary, but this is not very costly and makes the code more obviously correct. Fixes: nodejs#17840 PR-URL: nodejs#17863 Fixes: nodejs#17840 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>

PR-URL: nodejs#17620 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>

PR-URL: nodejs#17939 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Jon Moss <[email protected]> Reviewed-By: James M Snell <[email protected]>

The maxSessionMemory is a cap for the amount of memory an Http2Session is permitted to consume. If exceeded, new `Http2Stream` sessions will be rejected with an `ENHANCE_YOUR_CALM` error and existing `Http2Stream` instances that are still receiving headers will be terminated with an `ENHANCE_YOUR_CALM` error. Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17967 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Matteo Collina <[email protected]>

Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17968 Reviewed-By: Anna Henningsen <[email protected]>

Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17972 Reviewed-By: Weijia Wang <[email protected]> Reviewed-By: Jon Moss <[email protected]> Reviewed-By: Evan Lucas <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Tobias Nießen <[email protected]>

* verify protections against ping and settings flooding * Strictly handle and verify handling of unsolicited ping and settings frame acks. Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17969 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: Matteo Collina <[email protected]>

Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17911 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Daniel Bevenius <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Gibson Fahnestock <[email protected]> Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: Weijia Wang <[email protected]>

Adds the possibility to keep a strong persistent reference to a JS object while a `SetImmediate()` call is in effect. Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17183 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Franziska Hinkelmann <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]>

Calling into JS land from GC is not allowed, so delay the resolution of pending pings when a session is destroyed. Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17183 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Franziska Hinkelmann <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]>

Introduce an `Http2Scope` class that, when it goes out of scope, checks whether a write to the network is desired by nghttp2. If that is the case, schedule a write using `SetImmediate()` rather than a custom per-session libuv handle. Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17183 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Franziska Hinkelmann <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]>

Backport-PR-URL: #20456 PR-URL: #17406 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]> Backport-PR-URL: #18050

Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17406 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]> This is a significant cleanup and refactoring of the cleanup/close/destroy logic for Http2Stream and Http2Session. There are significant changes here in the timing and ordering of cleanup logic, JS apis. and various related necessary edits.

`nghttp2_stream_write_t` was not a necessary redirection layer and came with the cost of one additional allocation per stream write. Also, having both `nghttp2_stream_write` and `nghttp2_stream_write_t` as identifiers did not help with readability. Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17718 Reviewed-By: James M Snell <[email protected]>

- Only finish outgoing `WriteWrap`s once data has actually been passed to the underlying socket. - This makes HTTP2 streams respect backpressure - Use `DoTryWrite` as a shortcut for sending out as much of the data synchronously without blocking as possible - Use `NGHTTP2_DATA_FLAG_NO_COPY` to avoid copying DATA frame contents into nghttp2’s buffers before sending them out. Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17718 Reviewed-By: James M Snell <[email protected]>

Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17763 Refs: #17746 Reviewed-By: Matteo Collina <[email protected]>

Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17863 Fixes: #17840 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>

Keep a local handle as a reference to the JS `Http2Session` object so that it will not be garbage collected when inside an `Http2Scope`, because the presence of the latter usually indicates that further actions on the session object are expected. Strictly speaking, storing the `session_handle_` as a property on the scope object is not necessary, but this is not very costly and makes the code more obviously correct. Fixes: #17840 Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17863 Fixes: #17840 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>

Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17620 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>

Collect and report basic timing information about `Http2Session` and `Http2Stream` instances. Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17906 Refs: #17746 Reviewed-By: Matteo Collina <[email protected]>

Strictly limit the number of concurrent streams based on the current setting of the MAX_CONCURRENT_STREAMS setting Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #16766 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: Sebastiaan Deckers <[email protected]>

This commit also includes prerequisite error definitions from c75f87c and 1698c8e. Add support for sending and receiving ALTSVC frames. Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17917 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Tiancheng "Timothy" Gu <[email protected]> Reviewed-By: Matteo Collina <[email protected]>

Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17935 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: Sebastiaan Deckers <[email protected]> Reviewed-By: Tiancheng "Timothy" Gu <[email protected]>

Add new properties to `Http2Session` to identify alpnProtocol, and indicator about whether the session is TLS or not, and initial support for origin set (preparinng for `ORIGIN` frame support and the client-side `Pool` implementation. The `originSet` is the set of origins for which an `Http2Session` may be considered authoritative. Per the `ORIGIN` frame spec, the originSet is only valid on TLS connections, so this is only exposed when using a `TLSSocket`. Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17935 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: Sebastiaan Deckers <[email protected]> Reviewed-By: Tiancheng "Timothy" Gu <[email protected]>

Add a padding strategy option that makes a best attempt to ensure that total frame length for DATA and HEADERS frames are aligned on multiples of 8-bytes. Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17938 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Matteo Collina <[email protected]>

Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17942 Reviewed-By: Matteo Collina <[email protected]>

Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17954 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>

The maxSessionMemory is a cap for the amount of memory an Http2Session is permitted to consume. If exceeded, new `Http2Stream` sessions will be rejected with an `ENHANCE_YOUR_CALM` error and existing `Http2Stream` instances that are still receiving headers will be terminated with an `ENHANCE_YOUR_CALM` error. Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17967 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Matteo Collina <[email protected]>

Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17968 Reviewed-By: Anna Henningsen <[email protected]>

Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17972 Reviewed-By: Weijia Wang <[email protected]> Reviewed-By: Jon Moss <[email protected]> Reviewed-By: Evan Lucas <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Tobias Nießen <[email protected]>

* verify protections against ping and settings flooding * Strictly handle and verify handling of unsolicited ping and settings frame acks. Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17969 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: Matteo Collina <[email protected]>

Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17911 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Daniel Bevenius <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Gibson Fahnestock <[email protected]> Reviewed-By: Tobias Nießen <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Ruben Bridgewater <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: Weijia Wang <[email protected]>

nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. v9.x labels Jan 9, 2018

jasnell changed the title ~~Backport http2 v9.x~~ http2: backport changes to v9.x-staging Jan 9, 2018

jasnell added dont-land-on-v4.x http2 Issues or PRs related to the http2 subsystem. blocked PRs that are blocked by other issues or PRs. labels Jan 9, 2018

MylesBorins force-pushed the v9.x-staging branch from bd744b1 to f55a5e8 Compare January 9, 2018 08:20

MylesBorins force-pushed the backport-http2-v9.x branch from bf3c880 to 512fa41 Compare January 9, 2018 08:23

MylesBorins force-pushed the v9.x-staging branch from f55a5e8 to 30273d4 Compare January 9, 2018 10:02

addaleax and others added 12 commits January 9, 2018 05:09

http2: be sure to destroy the Http2Stream

504ee24

PR-URL: nodejs#17406 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]>

http2: convert Http2Settings to an AsyncWrap

293c2f9

PR-URL: nodejs#17763 Refs: nodejs#17746 Reviewed-By: Matteo Collina <[email protected]>

http2: fix compiling with --debug-http2

08478c1

PR-URL: nodejs#17863 Fixes: nodejs#17840 Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>

http2: implement ref() and unref() on client sessions

afa4a7c

PR-URL: nodejs#17620 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>

http2: remove duplicate words in comments

8d8dfea

PR-URL: nodejs#17939 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Jon Moss <[email protected]> Reviewed-By: James M Snell <[email protected]>

kjin pushed a commit to kjin/node that referenced this pull request May 1, 2018

http2: verify that a dependency cycle may exist

b3d09de

Backport-PR-URL: nodejs#18050 PR-URL: nodejs#17968 Reviewed-By: Anna Henningsen <[email protected]>

MylesBorins pushed a commit that referenced this pull request May 2, 2018

http2: convert Http2Settings to an AsyncWrap

0463221

Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17763 Refs: #17746 Reviewed-By: Matteo Collina <[email protected]>

MylesBorins pushed a commit that referenced this pull request May 2, 2018

http2: properly handle already closed stream error

9a6ea7e

Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17942 Reviewed-By: Matteo Collina <[email protected]>

MylesBorins pushed a commit that referenced this pull request May 2, 2018

doc: add docs for common/http2.js utility

35ee894

Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17942 Reviewed-By: Matteo Collina <[email protected]>

MylesBorins pushed a commit that referenced this pull request May 2, 2018

http2: verify that a dependency cycle may exist

a85518e

Backport-PR-URL: #18050 Backport-PR-URL: #20456 PR-URL: #17968 Reviewed-By: Anna Henningsen <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

http2: backport changes to v9.x-staging #18050

http2: backport changes to v9.x-staging #18050

jasnell commented Jan 9, 2018

targos commented Jan 9, 2018

jasnell commented Jan 9, 2018

mcollina commented Jan 9, 2018

jasnell commented Jan 9, 2018

MylesBorins commented Jan 9, 2018

targos commented Jan 9, 2018

MylesBorins commented Jan 9, 2018

http2: backport changes to v9.x-staging #18050

http2: backport changes to v9.x-staging #18050

Conversation

jasnell commented Jan 9, 2018

Checklist

Affected core subsystem(s)

targos commented Jan 9, 2018

jasnell commented Jan 9, 2018

mcollina commented Jan 9, 2018

jasnell commented Jan 9, 2018

MylesBorins commented Jan 9, 2018

targos commented Jan 9, 2018

MylesBorins commented Jan 9, 2018