Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

module: fix stat cache & simpler shebang #26266

Closed
wants to merge 5 commits into from

Conversation

BridgeAR
Copy link
Member

Please have a look at the commit messages for details.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

This simplifies the shebang function significantly. Before, it was
optimized for two characters input. Any module actually parsed should
however have more characters than just the shebang.
The performance stays the same as before.
The current caching logic broke by [0] because it used destructuring
on the module arguments. Since the exported property is a primitive
counting it up or down would not have any effect anymore in the module
that required that property.

The original implementation would cache all stat calls caused during
bootstrap. Afterwards it would clear the cache and lazy require calls
during runtime would create a new cascading cache for the then
loaded modules and clear the cache again.
This behavior is now restored. This is difficult to test without
exposing a lot of information and therfore the existing tests have
been removed (as they could not detect the issue).

With the broken implementation it caused each module compilation to
reset the cache and therefore minimizing the effect drastically.

[0] nodejs#19177
@nodejs-github-bot
Copy link
Collaborator

@BridgeAR sadly an error occured when I tried to trigger a build :(

lib/internal/modules/cjs/helpers.js Outdated Show resolved Hide resolved
@mscdex
Copy link
Contributor

mscdex commented Feb 23, 2019

CI is still unavailable, have you run module benchmarks?

@BridgeAR
Copy link
Member Author

@mscdex not for the stat cache alone. The shebang part stays the same performance wise and I measured general performance wins (including the stat cache change) in #25362. This is just a part of that PR as requested by @jdalton

@BridgeAR
Copy link
Member Author

BridgeAR commented Mar 1, 2019

@BridgeAR BridgeAR added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Mar 1, 2019
BridgeAR added a commit to BridgeAR/node that referenced this pull request Mar 1, 2019
This simplifies the shebang function significantly. Before, it was
optimized for two characters input. Any module actually parsed should
however have more characters than just the shebang.
The performance stays the same as before.

PR-URL: nodejs#26266
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: James M Snell <[email protected]>
BridgeAR added a commit to BridgeAR/node that referenced this pull request Mar 1, 2019
The current caching logic broke by [0] because it used destructuring
on the module arguments. Since the exported property is a primitive
counting it up or down would not have any effect anymore in the module
that required that property.

The original implementation would cache all stat calls caused during
bootstrap. Afterwards it would clear the cache and lazy require calls
during runtime would create a new cascading cache for the then
loaded modules and clear the cache again.
This behavior is now restored. This is difficult to test without
exposing a lot of information and therfore the existing tests have
been removed (as they could not detect the issue).

With the broken implementation it caused each module compilation to
reset the cache and therefore minimizing the effect drastically.

[0] nodejs#19177

PR-URL: nodejs#26266
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@BridgeAR
Copy link
Member Author

BridgeAR commented Mar 1, 2019

Thanks everyone for the reviews!

Landed in 3fce5cf, 410eb97 🎉

@BridgeAR BridgeAR closed this Mar 1, 2019
addaleax pushed a commit that referenced this pull request Mar 1, 2019
This simplifies the shebang function significantly. Before, it was
optimized for two characters input. Any module actually parsed should
however have more characters than just the shebang.
The performance stays the same as before.

PR-URL: #26266
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: James M Snell <[email protected]>
addaleax pushed a commit that referenced this pull request Mar 1, 2019
The current caching logic broke by [0] because it used destructuring
on the module arguments. Since the exported property is a primitive
counting it up or down would not have any effect anymore in the module
that required that property.

The original implementation would cache all stat calls caused during
bootstrap. Afterwards it would clear the cache and lazy require calls
during runtime would create a new cascading cache for the then
loaded modules and clear the cache again.
This behavior is now restored. This is difficult to test without
exposing a lot of information and therfore the existing tests have
been removed (as they could not detect the issue).

With the broken implementation it caused each module compilation to
reset the cache and therefore minimizing the effect drastically.

[0] #19177

PR-URL: #26266
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@BridgeAR BridgeAR mentioned this pull request Mar 4, 2019
@BridgeAR BridgeAR deleted the fix-stat-cache branch January 20, 2020 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants