Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v14.x backport] stream: simpler and faster Readable async iterator #34887

Closed

Conversation

ronag
Copy link
Member

@ronag ronag commented Aug 23, 2020

A backport of semver-major labelled PR. Was discussed with TSC who gave a go ahead to open a PR and try to gain approval from collaborators.

PR: #34035
Refs: #34680

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

Writable stream could emit 'finish' after 'close'.

PR-URL: #32933
Reviewed-By: Luigi Pinca [email protected]
Reviewed-By: Matteo Collina [email protected]

@ronag ronag added the stream Issues and PRs related to the stream subsystem. label Aug 23, 2020
@nodejs-github-bot nodejs-github-bot added build Issues and PRs related to build files or the CI. v14.x labels Aug 23, 2020
@ronag ronag changed the title stream: don't emit finish after close [v14.x backport] stream: don't emit finish after close Aug 23, 2020
@nodejs-github-bot

This comment has been minimized.

@ronag ronag changed the title [v14.x backport] stream: don't emit finish after close [v14.x backport] stream: simpler and faster Readable async iterator Aug 23, 2020
@ronag
Copy link
Member Author

ronag commented Aug 23, 2020

This has a dependency on #34103 which I had to include.

@ronag
Copy link
Member Author

ronag commented Aug 23, 2020

@MylesBorins There are two backports in this PR. Would I need to open separate PRs? The first commit doesn't make much sense to backport without the second one.

@nodejs-github-bot
Copy link
Collaborator

@ronag
Copy link
Member Author

ronag commented Aug 23, 2020

Weird test failure? AssertionError [ERR_ASSERTION]: benchmark file not running exactly one configuration in test:?

@mcollina mcollina added the tsc-agenda Issues and PRs to discuss during the meetings of the TSC. label Aug 23, 2020
Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm - I think this is the right thing to do long term.

@ronag
Copy link
Member Author

ronag commented Sep 1, 2020

@nodejs/streams

@nodejs-github-bot
Copy link
Collaborator

@richardlau
Copy link
Member

Weird test failure? AssertionError [ERR_ASSERTION]: benchmark file not running exactly one configuration in test:?

We're still seeing this in the CI on multiple platforms. Anyone able to take a look please?

@ronag
Copy link
Member Author

ronag commented Sep 3, 2020

Anyone able to take a look please?

Who can help with that? I have no clue.

@mhdawson
Copy link
Member

mhdawson commented Sep 3, 2020

Discussed in TSC meeting today, no objections raised by TSC members that were there. Myles, sees Matteo chiming that it is a good thing so comfortable that it land given that stream subsystem team members are signing off once ci is green. Removing the agenda tag.

@mhdawson mhdawson removed the tsc-agenda Issues and PRs to discuss during the meetings of the TSC. label Sep 3, 2020
@lundibundi
Copy link
Member

lundibundi commented Sep 3, 2020

Who can help with that? I have no clue.

@ronag I think you have to force a single configuration for your benchmark test in the test/benchmark/test-benchmark-streams.js
(see example in test/benchmark/test-benchmark-policy.js)

This is not the case since #31755.

@ronag ronag requested a review from mcollina October 14, 2020 10:19
@ronag ronag added the request-ci Add this label to start a Jenkins CI on a PR. label Oct 14, 2020
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 14, 2020
@nodejs-github-bot
Copy link
Collaborator

@ronag ronag requested a review from lpinca October 14, 2020 10:49
ronag added a commit to nxtedition/node that referenced this pull request Oct 14, 2020
Fixes some compatibility issues where it is expected
that for await stops reading when the stream is
destroyed.

Refs: nodejs#34887
@ronag
Copy link
Member Author

ronag commented Oct 14, 2020

This needs some more collaborator review to ensure we are confident with landing it in 14.x.

@BethGriggs
Copy link
Member

Just FYI in case it shows up in this PR, it looks like test-asan is failing on v14.x, i'm trying to figure that out in #35641.

@ronag
Copy link
Member Author

ronag commented Oct 14, 2020

@BethGriggs Green CI. Just needs another approval or two. Thanks for the ping!

@mcollina
Copy link
Member

What are the commits that are backported here? Have we got a references to those two PRs?

@ronag
Copy link
Member Author

ronag commented Oct 14, 2020

Yea I seem to have ruined the commit msg. The backported PR's are:

#34035
#35122

Also relevant:

#35640

@MylesBorins
Copy link
Contributor

@ronag it seems like multipel prs + commits have all merged into one here. I'm assuming that it will be a lot of work to split them up... so in lieu of that I think we could treat this as an entirely new change that avoids needing to backport the other change.

That does mean we will need at least 2 signoff before landing this.

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina
Copy link
Member

what is the diff vs what we have on master?

@ronag
Copy link
Member Author

ronag commented Oct 14, 2020

what is the diff vs what we have on master?

Once #35640 is merged, I believe they are exactly the same.

@MylesBorins
Copy link
Contributor

MylesBorins commented Oct 14, 2020

@mcollina @ronag . Should we rush getting this into the next 14? it will be the last release before LTS, planned to go out tomorrow

#35648

I'll be honest, considering these changes caused issues in the past I'm a bit nervous about including them just before flipping the LTS bit, but perhaps I am being too conservative. What do y'all think?

@mcollina
Copy link
Member

I think we would like to have this in v14 to simplify backporting.

@BethGriggs
Copy link
Member

I'll be honest, considering these changes caused issues in the past I'm a bit nervous about including them just before flipping the LTS bit, but perhaps I am being too conservative. What do y'all think?

FWIW, i'm somewhat leaning the opposite way, and feel that (for the same reasons) this change should land before v14.x is promoted to LTS. (Assuming that CITGM is happy.)

@MylesBorins
Copy link
Contributor

@ronag would you be able to rebase so we can make sure the asan test isn't broken by this PR

@MylesBorins
Copy link
Contributor

@nodejs/streams we still need 1 more sign off on this

@MylesBorins
Copy link
Contributor

I'm 99% sure the test failure here was broken on 14.x before this was opened and the PR just needs a rebase. I'm going to land this as soon as #35640 lands

MylesBorins pushed a commit that referenced this pull request Oct 15, 2020
Fixes some compatibility issues where it is expected
that for await stops reading when the stream is
destroyed.

Refs: #34887

PR-URL: #35640
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: James M Snell <[email protected]>
MylesBorins pushed a commit that referenced this pull request Oct 15, 2020
includes:

* stream: simpler and faster Readable async iterator
* stream: don't destroy on async iterator success
* stream: async iterator stop read if destroyed

PR-URL: #34887
Refs: #34035
Refs: #35122
Refs: #35640
Refs: #34680
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
@MylesBorins
Copy link
Contributor

MylesBorins commented Oct 15, 2020

Landed in 573410f

Hopefully I captures things accurately in the commit message

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver-minor PRs that contain new features and should be released in the next minor version. stream Issues and PRs related to the stream subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.