Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stream: improve Readable#from perf #50359

Merged
merged 1 commit into from
Oct 26, 2023

Conversation

rluvaton
Copy link
Member

@rluvaton rluvaton commented Oct 24, 2023

Benchmarks

Benchmark URL

15:39:36                                                                             confidence improvement accuracy (*)    (**)   (***)
15:39:36 streams/readable-from.js type='array' n=10000000                                   ***     23.20 %       ±9.27% ±12.34% ±16.09%
15:39:36 streams/readable-from.js type='async-generator' n=10000000                           *      2.28 %       ±1.83%  ±2.44%  ±3.17%
15:39:36 streams/readable-from.js type='sync-generator-with-async-values' n=10000000                 0.48 %       ±1.82%  ±2.42%  ±3.15%
15:39:36 streams/readable-from.js type='sync-generator-with-sync-values' n=10000000         ***     18.26 %       ±2.01%  ±2.68%  ±3.49%

@rluvaton rluvaton added stream Issues and PRs related to the stream subsystem. performance Issues and PRs related to the performance of Node.js. labels Oct 24, 2023
@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/streams

@nodejs-github-bot nodejs-github-bot added the needs-ci PRs that need a full CI run. label Oct 24, 2023
@rluvaton rluvaton added the request-ci Add this label to start a Jenkins CI on a PR. label Oct 24, 2023
@rluvaton

This comment was marked as outdated.

@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 24, 2023
@nodejs-github-bot
Copy link
Collaborator

@ronag
Copy link
Member

ronag commented Oct 24, 2023

Not sure I'm convinced the extra complexity is worth it. Let's see what the benchmark says.

@ronag
Copy link
Member

ronag commented Oct 24, 2023

@rluvaton when you start a benchmark ci in cases like this, add a filter e,g, from so not all benchmarks run.

@ronag
Copy link
Member

ronag commented Oct 24, 2023

https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1457/

@ronag
Copy link
Member

ronag commented Oct 24, 2023

11:20:15 streams/readable-from.js type='array' n=10000000                                            7.03 %       ±7.37% ±9.91% ±13.11%
11:20:15 streams/readable-from.js type='async-generator' n=10000000                           *      1.64 %       ±1.41% ±1.88%  ±2.45%
11:20:15 streams/readable-from.js type='sync-generator-with-async-values' n=10000000                 1.62 %       ±1.68% ±2.24%  ±2.92%
11:20:15 streams/readable-from.js type='sync-generator-with-sync-values' n=10000000         ***     11.09 %       ±1.82% ±2.42%  ±3.16%

lib/internal/streams/from.js Outdated Show resolved Hide resolved
lib/internal/streams/from.js Outdated Show resolved Hide resolved
@ronag
Copy link
Member

ronag commented Oct 24, 2023

I'm -0. There is a perf improvement but the maintainability cost is quite high.

@rluvaton
Copy link
Member Author

due to the removal of the async modifier I rerun the benchmark:
https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1458/

I'm -0. There is a perf improvement but the maintainability cost is quite high.

I'm on the fence as well...

@rluvaton
Copy link
Member Author

@ronag Updated the benchmarks...

@rluvaton rluvaton added the request-ci Add this label to start a Jenkins CI on a PR. label Oct 24, 2023
@H4ad H4ad added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Oct 24, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 24, 2023
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@benjamingr
Copy link
Member

I'm tired so if I don't make sense let me know - In the array case can't we just set the stream's buffer to ArrayPrototypeSlice(Input)? With ronag's bufferliist->array PR that could be ideal?

@rluvaton
Copy link
Member Author

I'm tired so if I don't make sense let me know - In the array case can't we just set the stream's buffer to ArrayPrototypeSlice(Input)? With ronag's bufferliist->array PR that could be ideal?

it would really improve the performance for sure but wouldn't it be dangerous as we need to make the stream reach the state where it would have data after init and everything

@nodejs-github-bot
Copy link
Collaborator

@ronag
Copy link
Member

ronag commented Oct 25, 2023

Wouldn't we skip a lot of state transitions that the user expects?

Not really?

@nodejs-github-bot
Copy link
Collaborator

@benjamingr
Copy link
Member

In that case this whole implementation would basically be:

Yeah that was my point in #50359 I think that would be way faster and probably semver-major

@rluvaton rluvaton added the commit-queue Add this label to land a pull request using GitHub Actions. label Oct 25, 2023
@rluvaton
Copy link
Member Author

In that case this whole implementation would basically be:

Yeah that was my point in #50359 I think that would be way faster and probably semver-major

Because it needs a semver major, I will create a new PR with the new code after that

@rluvaton rluvaton added the commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. label Oct 25, 2023
@rluvaton rluvaton added request-ci Add this label to start a Jenkins CI on a PR. and removed commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. labels Oct 25, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 25, 2023
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Oct 26, 2023
@nodejs-github-bot nodejs-github-bot merged commit 10d51e8 into nodejs:main Oct 26, 2023
50 checks passed
@nodejs-github-bot
Copy link
Collaborator

Landed in 10d51e8

@rluvaton rluvaton deleted the improve-from branch October 26, 2023 09:04
alexfernandez pushed a commit to alexfernandez/node that referenced this pull request Nov 1, 2023
PR-URL: nodejs#50359
Reviewed-By: Vinícius Lourenço Claro Cardoso <[email protected]>
Reviewed-By: Robert Nagy <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
targos pushed a commit that referenced this pull request Nov 11, 2023
PR-URL: #50359
Reviewed-By: Vinícius Lourenço Claro Cardoso <[email protected]>
Reviewed-By: Robert Nagy <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
UlisesGascon pushed a commit that referenced this pull request Dec 11, 2023
PR-URL: #50359
Reviewed-By: Vinícius Lourenço Claro Cardoso <[email protected]>
Reviewed-By: Robert Nagy <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
@UlisesGascon UlisesGascon mentioned this pull request Dec 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. needs-ci PRs that need a full CI run. performance Issues and PRs related to the performance of Node.js. stream Issues and PRs related to the stream subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants