Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

quic: delay destroying the readable until 'end' #35820

Closed

Conversation

mmomtchev
Copy link
Contributor

Delay destroying the readable until the end has been
received. This fixes the existing unit test that uses
an async iterator which won't read the last chunk
if the Readable has been destroyed.

Refs: 1d8ecd8
Fixes: #35789

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • commit message follows commit guidelines

Delay destroying the readable until the end has been
received. This fixes the existing unit test that uses
an async iterator which won't read the last chunk
if the Readable has been destroyed.

Refs: nodejs@1d8ecd8
Fixes: nodejs#35789
@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/quic

@nodejs-github-bot nodejs-github-bot added the quic Issues and PRs related to the QUIC implementation / HTTP/3. label Oct 26, 2020
@codecov-io
Copy link

Codecov Report

Merging #35820 into master will increase coverage by 0.00%.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #35820   +/-   ##
=======================================
  Coverage   87.90%   87.90%           
=======================================
  Files         477      477           
  Lines      113172   113172           
  Branches    25425    25428    +3     
=======================================
+ Hits        99480    99485    +5     
+ Misses       7992     7990    -2     
+ Partials     5700     5697    -3     
Impacted Files Coverage Δ
src/node_api.cc 69.64% <0.00%> (-0.56%) ⬇️
src/node_buffer.cc 70.11% <0.00%> (-0.48%) ⬇️
src/api/environment.cc 75.60% <0.00%> (-0.28%) ⬇️
lib/internal/util/inspect.js 95.58% <0.00%> (-0.15%) ⬇️
lib/fs.js 94.09% <0.00%> (ø)
lib/net.js 95.62% <0.00%> (ø)
src/node.h 93.33% <0.00%> (ø)
src/util.h 85.24% <0.00%> (ø)
lib/path.js 97.89% <0.00%> (ø)
lib/dgram.js 97.50% <0.00%> (ø)
... and 8 more

@Trott Trott requested a review from jasnell October 28, 2020 13:52
// Delay destroying the Readable until the end has been received
this.once('end', () => {
this.destroy();
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't it have autoDestroy enabled?

@@ -2673,7 +2673,14 @@ class QuicStream extends Duplex {

// Put the QuicStream into detached mode before calling destroy
this[kSetHandle]();
this.destroy();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think just removing this line would be the samething if autoDestroy is enabled.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also I think the current code is probably correct, kDestroy is not supposed to be a graceful shutdown? @jasnell?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

autoDestroy is indeed enabled
kDestroy() is called from onSessionClose() which is called from C++ QuicSession::Close(), it handles both errors and a normal close

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure that the code below is unnecessary and removing destroy() here might or might not be correct.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

node test/parallel/test-quic-simple-server-uni.js

You need a QUIC-enabled build

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test was working before you stopped sending the data from destroyed streams.
In QUIC you can mark a data chunk as being the final data chunk. In this case the client is supposed to close the connection upon receiving it. The QUIC code will immediately destroy the Readable in a nextTick handler. When using on('data') the user code will get a chance to read the last chunk, but when using the async iterator, and after your change, the Promise won't be resolved until the 'close' is processed.

Copy link
Contributor Author

@mmomtchev mmomtchev Oct 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jasnell said that a complete shake-up on that code was in the works, so he probably doesn't care that much 😄
But I had already debugged it before he said it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. Starting likely week after next this entire part of the code is going to be refactored entirely away from the Duplex interface. I don't have time to detail the specifics at the moment but will do so soon.

@@ -2673,7 +2673,14 @@ class QuicStream extends Duplex {

// Put the QuicStream into detached mode before calling destroy
this[kSetHandle]();
this.destroy();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also I think the current code is probably correct, kDestroy is not supposed to be a graceful shutdown? @jasnell?

@ronag
Copy link
Member

ronag commented Oct 28, 2020

@mmomtchev can you please also add a test so we can get a better understanding of what we are trying to fix here?

@ronag
Copy link
Member

ronag commented Oct 28, 2020

This looks like a more fundamental bug to me. IMO kDestroy should not be called until 'end' or at least destroy() should not be called in the graceful case. @jasnell

@mmomtchev
Copy link
Contributor Author

@ronag @jasnell How about pushing forward this PR as a temporary solution? It is a simple change that will fix a flaky OSX unit test that is failing very often (always maybe?)
@ronag destroy() is called in the graceful case almost everywhere - http, http2, tls...
I agree that it should not - that fix for the ECONNRESET issue showed why it is important to have a single generic and well-defined code-path for closing - but given that @jasnell is refactoring anyway, maybe this should be left out of this PR

@jasnell
Copy link
Member

jasnell commented Feb 1, 2021

Closing as no longer relevant since quic was removed.

@jasnell jasnell closed this Feb 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
quic Issues and PRs related to the QUIC implementation / HTTP/3.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

quic: async_iterator doesn't return the last chunk
6 participants