Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: ZipFS doesn't implement throwIfNoEntry which is slowing down tsc #4283

Closed
1 task
vire opened this issue Mar 28, 2022 · 4 comments · Fixed by #4475
Closed
1 task

[Bug]: ZipFS doesn't implement throwIfNoEntry which is slowing down tsc #4283

vire opened this issue Mar 28, 2022 · 4 comments · Fixed by #4475
Assignees
Labels
bug Something isn't working upholded Real issues without formal reproduction

Comments

@vire
Copy link

vire commented Mar 28, 2022

Self-service

  • I'd be willing to implement a fix

Describe the bug

When running tsc or eslint bins with typescript aware rules, this issue causes typescript program to resolve files for significant amount of time.

I did profile the process and flamegraph boils down to makeError function in pnp.cjs

image

see below on screenshot (the command ran for 25 seconds, but the reporting of non-existing files took most of the time)

image

when I console.log in makeError fn I get

rg ENOTDIR out | wc -l 
   230

and

rg ENOENT out | wc -l 
   59851

can be related to #2120

To reproduce

I could try to recreate a repo to reproduce but it would take some time

Environment

System:
    OS: macOS 12.1
    CPU: (8) arm64 Apple M1 Pro
    Memory: 86.58 MB / 16.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.14.0 - /private/var/folders/gr/8m03w_x95hxct7mpn9_gx8f40000gn/T/xfs-c7a7be7d/node
    Yarn: 3.2.0 - /private/var/folders/gr/8m03w_x95hxct7mpn9_gx8f40000gn/T/xfs-c7a7be7d/yarn
    npm: 8.3.1 - /var/folders/gr/8m03w_x95hxct7mpn9_gx8f40000gn/T/fnm_multishells/47019_1648458043892/bin/npm
  Managers:
    Homebrew: 3.4.3 - /opt/homebrew/bin/brew
    pip3: 22.0.2 - /opt/homebrew/bin/pip3
    RubyGems: 3.0.3.1 - /usr/bin/gem
  Utilities:
    Make: 3.81 - /usr/bin/make
    GCC: 4.2.1 - /usr/bin/gcc
    Git: 2.32.0 - /usr/bin/git
    Clang: 13.0.0 - /usr/bin/clang

Additional context

No response

@vire vire added the bug Something isn't working label Mar 28, 2022
@arcanis
Copy link
Member

arcanis commented Mar 28, 2022

That doesn't seem something about Yarn - TS just does a lot of "trial and error" searches in its resolution.

@arcanis arcanis added the waiting for feedback Will autoclose in a while unless more data are provided label Mar 28, 2022
@merceyz
Copy link
Member

merceyz commented Mar 28, 2022

We can fix this by implementing throwIfNoEntry which TypeScript uses to avoid the overhead of creating Errors, it's something I intend to implement but I haven't gotten around to it yet.

@merceyz merceyz added upholded Real issues without formal reproduction and removed waiting for feedback Will autoclose in a while unless more data are provided labels Mar 28, 2022
@merceyz merceyz changed the title [performance]: pnp ENOTDIR and ENOTDIR making typescript process slow [Bug]: PnP doesn't implement throwIfNoEntry which is slowing down tsc Mar 28, 2022
@vire
Copy link
Author

vire commented Mar 28, 2022

@arcanis @merceyz I did observe a really interesting behavior

having a workspaces monorepo and a single folder with own package.json and tsconfig.json, the time required for lint/compile is 1/2 compared when the root tsconfig.json is used + the amount of ENOTDIR and ENOENT is lower, therefore less calls to makeError

rg ENOTDIR out2 | wc -l
     114
rg ENOENT out2 | wc -l
   44116

I was thinking this issue relates to pnp feature when the TS calls to locate types in node_modules takes too long and eats a lot of CPU

@vire
Copy link
Author

vire commented Mar 28, 2022

an excerpt of the errors

ENOENT stat '/node_modules/@types/node
ENOENT stat '/node_modules/@types/react.d.ts'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/lodash/isEqual'
ENOENT stat '/node_modules/lodash/isEqual.ts'
ENOENT stat '/node_modules/lodash/isEqual.tsx'
ENOENT stat '/node_modules/lodash/isEqual.d.ts'
ENOENT stat '/node_modules/@types/lodash/isEqual'
ENOENT stat '/node_modules/@types/lodash/isEqual.ts'
ENOENT stat '/node_modules/@types/lodash/isEqual.tsx'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT open '/node_modules/chalk/source/package.json'
  • other 40k+ lines

@merceyz merceyz self-assigned this May 20, 2022
@merceyz merceyz changed the title [Bug]: PnP doesn't implement throwIfNoEntry which is slowing down tsc [Bug]: ZipFS doesn't implement throwIfNoEntry which is slowing down tsc May 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working upholded Real issues without formal reproduction
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants