Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce number of fs.stat call for files under node modules #52695

Closed
mjbvz opened this issue Feb 9, 2023 · 4 comments · Fixed by #52809
Closed

Reduce number of fs.stat call for files under node modules #52695

mjbvz opened this issue Feb 9, 2023 · 4 comments · Fixed by #52809
Assignees
Labels
Domain: Performance Reports of unusually slow behavior Experience Enhancement Noncontroversial enhancements Fix Available A PR has been opened for this issue Suggestion An idea for TypeScript

Comments

@mjbvz
Copy link
Contributor

mjbvz commented Feb 9, 2023

Bug Report

🔎 Search Terms

  • fs.stat
  • statSync
  • performance

Problem

While working on web project wide IntelliSense, I noticed that TS makes a number of fs.statSync calls for files stored under the global typings cache. At least some of these calls seem to be unnecessary, such as here where we check for files called mkdirp.ts, mkdirp.tsx, mkdirp.d.ts directly under node_modules:

/Users/matb/Library/Caches/typescript/4.9/node_modules/mkdirp
/Users/matb/Library/Caches/typescript/4.9/node_modules
/Users/matb/Library/Caches/typescript/4.9/node_modules/mkdirp.ts
/Users/matb/Library/Caches/typescript/4.9/node_modules/mkdirp.tsx
/Users/matb/Library/Caches/typescript/4.9/node_modules/mkdirp.d.ts
/Users/matb/Library/Caches/typescript/4.9/node_modules/mkdirp

I don't think these specific files would ever exist, would they?

These calls are relatively fast on desktop but do have more of an overhead on web. Even on desktop, I see around 250ms total spent on all the stat calls when starting TS Server in a simple project

Can we avoid making these calls?

@Andarist
Copy link
Contributor

Andarist commented Feb 9, 2023

I noticed this in September and asked @andrewbranch about this:

So Node's CJS resolution algorithm actually does look there, but its ESM one doesn't IIRC
And I think our ESM one incorrectly does, so it can be removed from ESM mode and type reference directives, but I don't want to change our CJS one, I think
Unless it could be shown to make a huge perf difference, then I think we'd consider it.

@andrewbranch
Copy link
Member

andrewbranch commented Feb 9, 2023

I thought about this more and decided having these files would be such an antipattern that I’m fine to deviate from Node’s CJS algorithm here. I pitched it to the team and nobody had objections. Just forgot to follow through. So yes, we can almost certainly avoid making these calls.

@andrewbranch andrewbranch added Suggestion An idea for TypeScript Experience Enhancement Noncontroversial enhancements Domain: Performance Reports of unusually slow behavior labels Feb 9, 2023
@andrewbranch andrewbranch self-assigned this Feb 9, 2023
@andrewbranch andrewbranch added this to the TypeScript 5.0.1 milestone Feb 9, 2023
@andrewbranch andrewbranch assigned sandersn and unassigned andrewbranch Feb 9, 2023
@mjbvz
Copy link
Contributor Author

mjbvz commented Feb 9, 2023

Similar case: when there's an import such as node:perf_hooks in your project, we end up stating these files:

stat /Users/matb/projects/vscode/node_modules/node:perf_hooks
stat /Users/matb/projects/vscode/node_modules/node:perf_hooks.ts
stat /Users/matb/projects/vscode/node_modules/node:perf_hooks.tsx
stat /Users/matb/projects/vscode/node_modules/node:perf_hooks.d.ts
stat /Users/matb/projects/vscode/node_modules/@types/node:perf_hooks
stat /Users/matb/projects/vscode/node_modules/@types/node:perf_hooks.d.ts

Can we skip these checks for node:___ package names?

@andrewbranch
Copy link
Member

Yeah, I think so. We probably shouldn’t do directory searches for anything that looks like an absolute URI (#35749)

sandersn added a commit to sandersn/TypeScript that referenced this issue Feb 16, 2023
or in @types. Although the node module resolution algorithm looks for
.js files there, they would never be there in correctly configured
node_modules. So it should be safe to skip *.ts, *.js, *.tsx, etc.

Also skips looking for files directly in node_modules/@types.

Fixes microsoft#52695
@typescript-bot typescript-bot added the Fix Available A PR has been opened for this issue label Feb 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Domain: Performance Reports of unusually slow behavior Experience Enhancement Noncontroversial enhancements Fix Available A PR has been opened for this issue Suggestion An idea for TypeScript
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants