stream: reset awaitDrain after manual .resume() #7160
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Reset the `readableState.awaitDrain` counter after manual calls to `.resume()`. What might happen otherwise is that a slow consumer at the end of the pipe could end up stalling the piping in the following scenario: 1. The writable stream indicates that its buffer is full. 2. This leads the readable stream to `pause()` and increase its `awaitDrain` counter, which will be decreased by the writable’s next `drain` event. 3. Something calls `.resume()` manually. 4. The readable continues to pipe to the writable, but once again the writable stream indicates that the buffer is full. 5. The `awaitDrain` counter is thus increased again, but since it has now been increased twice for a single piping destination, the next `drain` event will not be able to reset `awaitDrain` to zero. 6. The pipe is stalled and no data is passed along anymore. The solution in this commit is to reset the `awaitDrain` counter to zero when `resume()` is called. Fixes: nodejs#7159
|
I think this should be ok to backport, but we should probably check CIGTM before that. cc @calvinmetcalf LGTM |
|
LGTM if @nodejs/streams folks are happy. |
4 tasks
3 tasks
|
@thealphanerd since you self-assigned this… anything in particular you’d like to see before landing? |
|
Doesn't look to be anything to controversial from my perspective |
|
@calvinmetcalf I'm assuming you did not mean to close this 😄 |
|
@addaleax Nothing I need to see before landing, assigned to myself to keep an eye on it for LTS |
|
@thealphanerd you are correct sorry foks |
|
as long as @mcollina is happy i'm happy so 👍 |
addaleax
added a commit
that referenced
this pull request
Jun 13, 2016
Reset the `readableState.awaitDrain` counter after manual calls to `.resume()`. What might happen otherwise is that a slow consumer at the end of the pipe could end up stalling the piping in the following scenario: 1. The writable stream indicates that its buffer is full. 2. This leads the readable stream to `pause()` and increase its `awaitDrain` counter, which will be decreased by the writable’s next `drain` event. 3. Something calls `.resume()` manually. 4. The readable continues to pipe to the writable, but once again the writable stream indicates that the buffer is full. 5. The `awaitDrain` counter is thus increased again, but since it has now been increased twice for a single piping destination, the next `drain` event will not be able to reset `awaitDrain` to zero. 6. The pipe is stalled and no data is passed along anymore. The solution in this commit is to reset the `awaitDrain` counter to zero when `resume()` is called. Fixes: #7159 PR-URL: #7160 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: James M Snell <[email protected]>
|
Landed in e2e615e, thanks for the reviews and comments, everyone! |
evanlucas
pushed a commit
that referenced
this pull request
Jun 16, 2016
Reset the `readableState.awaitDrain` counter after manual calls to `.resume()`. What might happen otherwise is that a slow consumer at the end of the pipe could end up stalling the piping in the following scenario: 1. The writable stream indicates that its buffer is full. 2. This leads the readable stream to `pause()` and increase its `awaitDrain` counter, which will be decreased by the writable’s next `drain` event. 3. Something calls `.resume()` manually. 4. The readable continues to pipe to the writable, but once again the writable stream indicates that the buffer is full. 5. The `awaitDrain` counter is thus increased again, but since it has now been increased twice for a single piping destination, the next `drain` event will not be able to reset `awaitDrain` to zero. 6. The pipe is stalled and no data is passed along anymore. The solution in this commit is to reset the `awaitDrain` counter to zero when `resume()` is called. Fixes: #7159 PR-URL: #7160 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: James M Snell <[email protected]>
Merged
abresas
added a commit
to balena-io-modules/open-balena-base
that referenced
this pull request
Jun 20, 2016
fixes streams error nodejs/node#7160
abresas
added a commit
to balena-io-modules/open-balena-base
that referenced
this pull request
Jun 20, 2016
fixes streams error nodejs/node#7160
MylesBorins
pushed a commit
that referenced
this pull request
Jun 23, 2016
Reset the `readableState.awaitDrain` counter after manual calls to `.resume()`. What might happen otherwise is that a slow consumer at the end of the pipe could end up stalling the piping in the following scenario: 1. The writable stream indicates that its buffer is full. 2. This leads the readable stream to `pause()` and increase its `awaitDrain` counter, which will be decreased by the writable’s next `drain` event. 3. Something calls `.resume()` manually. 4. The readable continues to pipe to the writable, but once again the writable stream indicates that the buffer is full. 5. The `awaitDrain` counter is thus increased again, but since it has now been increased twice for a single piping destination, the next `drain` event will not be able to reset `awaitDrain` to zero. 6. The pipe is stalled and no data is passed along anymore. The solution in this commit is to reset the `awaitDrain` counter to zero when `resume()` is called. Fixes: #7159 PR-URL: #7160 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: James M Snell <[email protected]>
Merged
MylesBorins
pushed a commit
that referenced
this pull request
Jun 24, 2016
Reset the `readableState.awaitDrain` counter after manual calls to `.resume()`. What might happen otherwise is that a slow consumer at the end of the pipe could end up stalling the piping in the following scenario: 1. The writable stream indicates that its buffer is full. 2. This leads the readable stream to `pause()` and increase its `awaitDrain` counter, which will be decreased by the writable’s next `drain` event. 3. Something calls `.resume()` manually. 4. The readable continues to pipe to the writable, but once again the writable stream indicates that the buffer is full. 5. The `awaitDrain` counter is thus increased again, but since it has now been increased twice for a single piping destination, the next `drain` event will not be able to reset `awaitDrain` to zero. 6. The pipe is stalled and no data is passed along anymore. The solution in this commit is to reset the `awaitDrain` counter to zero when `resume()` is called. Fixes: #7159 PR-URL: #7160 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: James M Snell <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
Jun 24, 2016
Reset the `readableState.awaitDrain` counter after manual calls to `.resume()`. What might happen otherwise is that a slow consumer at the end of the pipe could end up stalling the piping in the following scenario: 1. The writable stream indicates that its buffer is full. 2. This leads the readable stream to `pause()` and increase its `awaitDrain` counter, which will be decreased by the writable’s next `drain` event. 3. Something calls `.resume()` manually. 4. The readable continues to pipe to the writable, but once again the writable stream indicates that the buffer is full. 5. The `awaitDrain` counter is thus increased again, but since it has now been increased twice for a single piping destination, the next `drain` event will not be able to reset `awaitDrain` to zero. 6. The pipe is stalled and no data is passed along anymore. The solution in this commit is to reset the `awaitDrain` counter to zero when `resume()` is called. Fixes: #7159 PR-URL: #7160 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: James M Snell <[email protected]>
MylesBorins
pushed a commit
that referenced
this pull request
Jun 28, 2016
This LTS release comes with 89 commits. This includes 46 commits that
are docs related, 11 commits that are test related, 8 commits that are
build related, and 4 commits that are benchmark related.
Notable Changes:
- debugger:
- All properties of an array (aside from length) can now be printed
in the repl (cjihrig)
#6448
- npm:
- Upgrade npm to 2.15.8 (Rebecca Turner)
#7412
- stream:
- Fix for a bug that became more prevalent with the stream changes
that landed in v4.4.5. (Anna Henningsen)
#7160
- V8:
- Fix for a bug in crankshaft that was causing crashes on arm64
(Myles Borins)
#7442
- Add missing classes to postmortem info such as JSMap and JSSet
(evan.lucas)
#3792
MylesBorins
pushed a commit
that referenced
this pull request
Jun 28, 2016
This LTS release comes with 89 commits. This includes 46 commits that
are docs related, 11 commits that are test related, 8 commits that are
build related, and 4 commits that are benchmark related.
Notable Changes:
- debugger:
- All properties of an array (aside from length) can now be printed
in the repl (cjihrig)
#6448
- npm:
- Upgrade npm to 2.15.8 (Rebecca Turner)
#7412
- stream:
- Fix for a bug that became more prevalent with the stream changes
that landed in v4.4.5. (Anna Henningsen)
#7160
- V8:
- Fix for a bug in crankshaft that was causing crashes on arm64
(Myles Borins)
#7442
- Add missing classes to postmortem info such as JSMap and JSSet
(evan.lucas)
#3792
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Checklist
Affected core subsystem(s)
stream
Description of change
Reset the
readableState.awaitDraincounter after manual calls to.resume().What might happen otherwise is that a slow consumer at the end of the pipe could end up stalling the piping in the following scenario:
pause()and increase itsawaitDraincounter, which will be decreased by the writable’s nextdrainevent..resume()manually.awaitDraincounter is thus increased again, but since it has now been increased twice for a single piping destination, the nextdrainevent will not be able to resetawaitDrainto zero.The solution in this commit is to reset the
awaitDraincounter to zero whenresume()is called.Fixes: #7159
This bug is has been around at least since 0.12, and I don’t currently know why it never popped up before #2325 when the
awaitDrainmechanism worked like it does today./cc @nodejs/streams