Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[v14.x backport] stream: simpler and faster Readable async iterator #34887

Closed
wants to merge 6 commits into from
Closed

[v14.x backport] stream: simpler and faster Readable async iterator #34887

wants to merge 6 commits into from
+237 −271

Conversation

ronag
Copy link
Member

@ronag ronag commented Aug 23, 2020 •
edited
Loading

A backport of semver-major labelled PR. Was discussed with TSC who gave a go ahead to open a PR and try to gain approval from collaborators.

PR: #34035
Refs: #34680

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

Writable stream could emit 'finish' after 'close'.

PR-URL: #32933
Reviewed-By: Luigi Pinca [email protected]
Reviewed-By: Matteo Collina [email protected]

@ronag ronag added the stream Issues and PRs related to the stream subsystem. label Aug 23, 2020
@nodejs-github-bot nodejs-github-bot added build Issues and PRs related to build files or the CI. v14.x labels Aug 23, 2020
@ronag ronag changed the title stream: don't emit finish after close [v14.x backport] stream: don't emit finish after close Aug 23, 2020
@nodejs-github-bot

This comment has been minimized.

Sign in to view
@ronag ronag changed the title [v14.x backport] stream: don't emit finish after close [v14.x backport] stream: simpler and faster Readable async iterator Aug 23, 2020
@ronag
Copy link
Member Author

ronag commented Aug 23, 2020 •
edited
Loading

This has a dependency on #34103 which I had to include.

@ronag
Copy link
Member Author

ronag commented Aug 23, 2020

@MylesBorins There are two backports in this PR. Would I need to open separate PRs? The first commit doesn't make much sense to backport without the second one.

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/32898/

@ronag
Copy link
Member Author

ronag commented Aug 23, 2020

Weird test failure? AssertionError [ERR_ASSERTION]: benchmark file not running exactly one configuration in test:?

@mcollina mcollina added the tsc-agenda Issues and PRs to discuss during the meetings of the TSC. label Aug 23, 2020
mcollina
mcollina approved these changes Aug 23, 2020
View reviewed changes
Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm - I think this is the right thing to do long term.

This was referenced Aug 24, 2020
Closed
Closed
@ronag
Copy link
Member Author

ronag commented Sep 1, 2020

@nodejs/streams

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/32990/

@richardlau
Copy link
Member

Weird test failure? AssertionError [ERR_ASSERTION]: benchmark file not running exactly one configuration in test:?

We're still seeing this in the CI on multiple platforms. Anyone able to take a look please?

@ronag
Copy link
Member Author

ronag commented Sep 3, 2020

Anyone able to take a look please?

Who can help with that? I have no clue.

@mhdawson
Copy link
Member

mhdawson commented Sep 3, 2020

Discussed in TSC meeting today, no objections raised by TSC members that were there. Myles, sees Matteo chiming that it is a good thing so comfortable that it land given that stream subsystem team members are signing off once ci is green. Removing the agenda tag.

@mhdawson mhdawson removed the tsc-agenda Issues and PRs to discuss during the meetings of the TSC. label Sep 3, 2020
@lundibundi
Copy link
Member

lundibundi commented Sep 3, 2020 •
edited
Loading

Who can help with that? I have no clue.

@ronag I think you have to force a single configuration for your benchmark test in the test/benchmark/test-benchmark-streams.js
(see example in test/benchmark/test-benchmark-policy.js)

This is not the case since #31755.

@ronag ronag requested a review from mcollina October 14, 2020 10:19
@ronag ronag added the request-ci Add this label to start a Jenkins CI on a PR. label Oct 14, 2020
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 14, 2020
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/33613/

@ronag ronag requested a review from lpinca October 14, 2020 10:49
ronag added a commit to nxtedition/node that referenced this pull request Oct 14, 2020
@ronag
stream: async iterator stop read if destroyed
2c9432b 
Fixes some compatibility issues where it is expected
that for await stops reading when the stream is
destroyed.

Refs: nodejs#34887
@ronag ronag mentioned this pull request Oct 14, 2020
Closed
4 tasks
@ronag
Copy link
Member Author

ronag commented Oct 14, 2020

This needs some more collaborator review to ensure we are confident with landing it in 14.x.

@BethGriggs
Copy link
Member

Just FYI in case it shows up in this PR, it looks like test-asan is failing on v14.x, i'm trying to figure that out in #35641.

@ronag
Copy link
Member Author

ronag commented Oct 14, 2020 •
edited
Loading

@BethGriggs Green CI. Just needs another approval or two. Thanks for the ping!

@mcollina
Copy link
Member

What are the commits that are backported here? Have we got a references to those two PRs?

@ronag
Copy link
Member Author

ronag commented Oct 14, 2020 •
edited
Loading

Yea I seem to have ruined the commit msg. The backported PR's are:

#34035
#35122

Also relevant:

#35640

@MylesBorins
Copy link
Contributor

@ronag it seems like multipel prs + commits have all merged into one here. I'm assuming that it will be a lot of work to split them up... so in lieu of that I think we could treat this as an entirely new change that avoids needing to backport the other change.

That does mean we will need at least 2 signoff before landing this.

mcollina
mcollina approved these changes Oct 14, 2020
View reviewed changes
Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina
Copy link
Member

what is the diff vs what we have on master?

@ronag
Copy link
Member Author

ronag commented Oct 14, 2020 •
edited
Loading

what is the diff vs what we have on master?

Once #35640 is merged, I believe they are exactly the same.

@MylesBorins
Copy link
Contributor

MylesBorins commented Oct 14, 2020 •
edited
Loading

@mcollina @ronag . Should we rush getting this into the next 14? it will be the last release before LTS, planned to go out tomorrow

#35648

I'll be honest, considering these changes caused issues in the past I'm a bit nervous about including them just before flipping the LTS bit, but perhaps I am being too conservative. What do y'all think?

@mcollina
Copy link
Member

I think we would like to have this in v14 to simplify backporting.

@BethGriggs
Copy link
Member

I'll be honest, considering these changes caused issues in the past I'm a bit nervous about including them just before flipping the LTS bit, but perhaps I am being too conservative. What do y'all think?

FWIW, i'm somewhat leaning the opposite way, and feel that (for the same reasons) this change should land before v14.x is promoted to LTS. (Assuming that CITGM is happy.)

@MylesBorins
Copy link
Contributor

@ronag would you be able to rebase so we can make sure the asan test isn't broken by this PR

@MylesBorins
Copy link
Contributor

@nodejs/streams we still need 1 more sign off on this

@MylesBorins
Copy link
Contributor

I'm 99% sure the test failure here was broken on 14.x before this was opened and the PR just needs a rebase. I'm going to land this as soon as #35640 lands

MylesBorins pushed a commit that referenced this pull request Oct 15, 2020
@ronag @MylesBorins
stream: async iterator stop read if destroyed
1d8ecd8 
Fixes some compatibility issues where it is expected
that for await stops reading when the stream is
destroyed.

Refs: #34887

PR-URL: #35640
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: James M Snell <[email protected]>
MylesBorins pushed a commit that referenced this pull request Oct 15, 2020
@ronag @MylesBorins
stream: multiple stream backports
573410f 
includes:

* stream: simpler and faster Readable async iterator
* stream: don't destroy on async iterator success
* stream: async iterator stop read if destroyed

PR-URL: #34887
Refs: #34035
Refs: #35122
Refs: #35640
Refs: #34680
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
@MylesBorins
Copy link
Contributor

MylesBorins commented Oct 15, 2020 •
edited
Loading

Landed in 573410f

Hopefully I captures things accurately in the commit message

@christhegrand christhegrand mentioned this pull request Mar 23, 2021
Closed
@targos targos mentioned this pull request May 16, 2021
Closed
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@richardlau richardlau richardlau left review comments

@vweevers vweevers vweevers left review comments

@lpinca lpinca lpinca approved these changes

@mcollina mcollina mcollina approved these changes

@benjamingr benjamingr Awaiting requested review from benjamingr

Assignees
No one assigned
Labels
semver-minor PRs that contain new features and should be released in the next minor version. stream Issues and PRs related to the stream subsystem.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.