Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Destructure common factory methods in the parser. #52920

Merged
merged 5 commits into from
Feb 24, 2023

Conversation

DanielRosenwasser
Copy link
Member

@DanielRosenwasser DanielRosenwasser commented Feb 22, 2023

In #35282, we switched over to using factory functions within the parser - this ensured that nodes of the same type were more uniformly initialized, but with a severe slowdown in the parser. This presumably came from several new indirections like object member access and method invocation costs.

This PR gets some performance savings back from directly destructuring every factory method from the parser's factory. It ends up being a relatively small amount saved, but I figured we might want to discuss it anyway.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Feb 22, 2023
@DanielRosenwasser
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Feb 22, 2023

Heya @DanielRosenwasser, I've started to run the perf test suite on this PR at 41257fe. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..52920
Metric main 52920 Delta Best Worst p-value
Angular - node (v18.10.0, x64)
Memory used 359,541k (± 0.02%) 359,606k (± 0.01%) ~ 359,555k 359,664k p=0.093 n=6
Parse Time 3.50s (± 1.15%) 3.51s (± 0.86%) ~ 3.47s 3.55s p=0.518 n=6
Bind Time 1.13s (± 0.67%) 1.12s (± 0.46%) ~ 1.12s 1.13s p=0.247 n=6
Check Time 8.69s (± 0.47%) 8.66s (± 0.60%) ~ 8.59s 8.72s p=0.373 n=6
Emit Time 7.45s (± 0.25%) 7.44s (± 0.42%) ~ 7.39s 7.48s p=0.627 n=6
Total Time 20.77s (± 0.38%) 20.74s (± 0.38%) ~ 20.65s 20.86s p=0.514 n=6
Compiler-Unions - node (v18.10.0, x64)
Memory used 189,701k (± 0.04%) 191,681k (± 1.54%) ~ 189,586k 195,591k p=0.065 n=6
Parse Time 1.49s (± 0.37%) 1.47s (± 0.93%) -0.03s (- 1.90%) 1.45s 1.49s p=0.008 n=6
Bind Time 0.78s (± 1.08%) 0.77s (± 0.82%) ~ 0.76s 0.78s p=0.226 n=6
Check Time 9.40s (± 0.98%) 9.39s (± 0.69%) ~ 9.33s 9.50s p=1.000 n=6
Emit Time 2.77s (± 1.28%) 2.74s (± 0.75%) ~ 2.72s 2.77s p=0.087 n=6
Total Time 14.44s (± 0.54%) 14.37s (± 0.46%) ~ 14.27s 14.47s p=0.078 n=6
Monaco - node (v18.10.0, x64)
Memory used 343,864k (± 0.01%) 343,949k (± 0.01%) +86k (+ 0.02%) 343,926k 343,985k p=0.005 n=6
Parse Time 2.63s (± 1.35%) 2.64s (± 0.52%) ~ 2.61s 2.65s p=0.571 n=6
Bind Time 1.01s (± 0.40%) 1.00s (± 1.50%) ~ 0.98s 1.02s p=0.498 n=6
Check Time 7.00s (± 0.48%) 6.99s (± 0.27%) ~ 6.97s 7.01s p=0.360 n=6
Emit Time 4.23s (± 1.08%) 4.21s (± 0.73%) ~ 4.18s 4.26s p=0.518 n=6
Total Time 14.86s (± 0.60%) 14.84s (± 0.16%) ~ 14.81s 14.87s p=0.809 n=6
TFS - node (v18.10.0, x64)
Memory used 299,854k (± 0.01%) 299,906k (± 0.01%) +52k (+ 0.02%) 299,869k 299,922k p=0.013 n=6
Parse Time 2.04s (± 0.98%) 1.99s (± 1.41%) -0.05s (- 2.21%) 1.95s 2.03s p=0.015 n=6
Bind Time 1.14s (± 0.36%) 1.14s (± 0.55%) ~ 1.13s 1.15s p=0.673 n=6
Check Time 6.50s (± 0.33%) 6.49s (± 0.71%) ~ 6.42s 6.56s p=0.936 n=6
Emit Time 3.83s (± 0.39%) 3.88s (± 0.46%) +0.05s (+ 1.22%) 3.86s 3.91s p=0.005 n=6
Total Time 13.51s (± 0.21%) 13.51s (± 0.32%) ~ 13.45s 13.57s p=0.936 n=6
material-ui - node (v18.10.0, x64)
Memory used 476,409k (± 0.00%) 476,363k (± 0.00%) -46k (- 0.01%) 476,340k 476,399k p=0.008 n=6
Parse Time 3.15s (± 0.40%) 3.12s (± 1.16%) ~ 3.08s 3.18s p=0.106 n=6
Bind Time 0.91s (± 0.57%) 0.91s (± 0.60%) ~ 0.90s 0.91s p=0.640 n=6
Check Time 17.14s (± 0.97%) 17.09s (± 0.38%) ~ 17.00s 17.18s p=0.575 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 21.20s (± 0.81%) 21.12s (± 0.34%) ~ 20.99s 21.20s p=0.419 n=6
xstate - node (v18.10.0, x64)
Memory used 548,167k (± 0.02%) 548,291k (± 0.02%) ~ 548,137k 548,486k p=0.093 n=6
Parse Time 4.07s (± 0.79%) 4.03s (± 1.06%) ~ 3.99s 4.08s p=0.125 n=6
Bind Time 1.67s (± 0.33%) 1.68s (± 1.04%) ~ 1.65s 1.70s p=0.866 n=6
Check Time 2.75s (± 1.01%) 2.76s (± 0.58%) ~ 2.74s 2.78s p=0.744 n=6
Emit Time 0.08s (± 4.99%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=0.405 n=6
Total Time 8.59s (± 0.57%) 8.54s (± 0.38%) ~ 8.51s 8.59s p=0.106 n=6
Angular - node (v16.17.1, x64)
Memory used 358,876k (± 0.00%) 358,950k (± 0.01%) +74k (+ 0.02%) 358,921k 358,975k p=0.005 n=6
Parse Time 3.72s (± 0.53%) 3.69s (± 0.24%) -0.03s (- 0.90%) 3.68s 3.70s p=0.019 n=6
Bind Time 1.19s (± 0.92%) 1.19s (± 0.34%) ~ 1.19s 1.20s p=0.445 n=6
Check Time 9.46s (± 0.33%) 9.41s (± 0.51%) ~ 9.36s 9.49s p=0.078 n=6
Emit Time 7.94s (± 0.97%) 7.91s (± 0.74%) ~ 7.84s 8.00s p=0.421 n=6
Total Time 22.32s (± 0.43%) 22.19s (± 0.43%) ~ 22.07s 22.32s p=0.078 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 191,955k (± 0.71%) 191,970k (± 0.70%) ~ 191,347k 194,697k p=0.810 n=6
Parse Time 1.57s (± 0.87%) 1.55s (± 0.67%) -0.03s (- 1.69%) 1.53s 1.56s p=0.014 n=6
Bind Time 0.82s (± 0.50%) 0.82s (± 0.63%) ~ 0.82s 0.83s p=0.114 n=6
Check Time 10.11s (± 0.27%) 10.09s (± 0.58%) ~ 10.04s 10.20s p=0.170 n=6
Emit Time 3.00s (± 1.01%) 2.98s (± 0.92%) ~ 2.94s 3.01s p=0.228 n=6
Total Time 15.50s (± 0.34%) 15.44s (± 0.53%) ~ 15.34s 15.58s p=0.128 n=6
Monaco - node (v16.17.1, x64)
Memory used 343,080k (± 0.00%) 343,175k (± 0.00%) +95k (+ 0.03%) 343,153k 343,192k p=0.005 n=6
Parse Time 2.80s (± 0.32%) 2.78s (± 0.95%) ~ 2.74s 2.81s p=0.284 n=6
Bind Time 1.08s (± 0.50%) 1.08s (± 0.75%) ~ 1.07s 1.09s p=0.859 n=6
Check Time 7.70s (± 0.42%) 7.66s (± 0.38%) ~ 7.63s 7.71s p=0.054 n=6
Emit Time 4.44s (± 0.63%) 4.42s (± 0.47%) ~ 4.40s 4.46s p=0.366 n=6
Total Time 16.02s (± 0.32%) 15.95s (± 0.25%) -0.07s (- 0.46%) 15.91s 16.01s p=0.043 n=6
TFS - node (v16.17.1, x64)
Memory used 299,214k (± 0.01%) 299,257k (± 0.00%) +42k (+ 0.01%) 299,236k 299,268k p=0.013 n=6
Parse Time 2.18s (± 0.69%) 2.15s (± 0.46%) -0.03s (- 1.60%) 2.13s 2.16s p=0.004 n=6
Bind Time 1.23s (± 1.22%) 1.24s (± 0.99%) ~ 1.22s 1.25s p=0.931 n=6
Check Time 7.18s (± 0.53%) 7.17s (± 0.38%) ~ 7.15s 7.22s p=0.746 n=6
Emit Time 4.37s (± 0.77%) 4.31s (± 0.49%) -0.06s (- 1.30%) 4.29s 4.35s p=0.015 n=6
Total Time 14.96s (± 0.41%) 14.87s (± 0.24%) -0.09s (- 0.62%) 14.82s 14.91s p=0.030 n=6
material-ui - node (v16.17.1, x64)
Memory used 475,754k (± 0.01%) 475,701k (± 0.02%) ~ 475,606k 475,832k p=0.378 n=6
Parse Time 3.33s (± 0.41%) 3.30s (± 0.19%) -0.03s (- 1.00%) 3.29s 3.31s p=0.005 n=6
Bind Time 0.96s (± 0.42%) 0.96s (± 0.85%) ~ 0.95s 0.97s p=0.206 n=6
Check Time 18.14s (± 0.83%) 18.10s (± 0.42%) ~ 17.99s 18.20s p=1.000 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.44s (± 0.61%) 22.36s (± 0.35%) ~ 22.25s 22.47s p=0.293 n=6
xstate - node (v16.17.1, x64)
Memory used 545,746k (± 0.02%) 545,999k (± 0.04%) +253k (+ 0.05%) 545,749k 546,280k p=0.031 n=6
Parse Time 4.25s (± 0.33%) 4.23s (± 0.35%) ~ 4.22s 4.25s p=0.072 n=6
Bind Time 1.74s (± 0.43%) 1.76s (± 0.23%) +0.02s (+ 0.96%) 1.75s 1.76s p=0.005 n=6
Check Time 3.00s (± 0.54%) 2.98s (± 0.39%) ~ 2.97s 3.00s p=0.119 n=6
Emit Time 0.09s (± 0.00%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=1.000 n=6
Total Time 9.08s (± 0.31%) 9.07s (± 0.25%) ~ 9.05s 9.11s p=0.511 n=6
Angular - node (v14.15.1, x64)
Memory used 352,600k (± 0.00%) 352,464k (± 0.05%) ~ 352,316k 352,667k p=0.378 n=6
Parse Time 3.71s (± 0.67%) 3.69s (± 0.61%) ~ 3.65s 3.71s p=0.324 n=6
Bind Time 1.25s (± 0.67%) 1.22s (± 0.68%) -0.02s (- 1.61%) 1.22s 1.24s p=0.010 n=6
Check Time 9.71s (± 0.57%) 9.70s (± 0.34%) ~ 9.64s 9.74s p=0.809 n=6
Emit Time 8.37s (± 1.10%) 8.42s (± 1.75%) ~ 8.25s 8.62s p=0.873 n=6
Total Time 23.04s (± 0.69%) 23.04s (± 0.76%) ~ 22.86s 23.29s p=0.810 n=6
Compiler-Unions - node (v14.15.1, x64)
Memory used 186,614k (± 0.01%) 186,637k (± 0.02%) ~ 186,608k 186,685k p=0.128 n=6
Parse Time 1.59s (± 1.08%) 1.55s (± 0.71%) -0.04s (- 2.41%) 1.53s 1.56s p=0.005 n=6
Bind Time 0.84s (± 0.97%) 0.84s (± 0.00%) ~ 0.84s 0.84s p=0.290 n=6
Check Time 10.18s (± 0.47%) 10.16s (± 0.32%) ~ 10.12s 10.21s p=0.466 n=6
Emit Time 3.11s (± 0.62%) 3.09s (± 0.64%) ~ 3.07s 3.12s p=0.211 n=6
Total Time 15.71s (± 0.39%) 15.64s (± 0.26%) -0.07s (- 0.46%) 15.61s 15.72s p=0.036 n=6
Monaco - node (v14.15.1, x64)
Memory used 338,054k (± 0.01%) 338,130k (± 0.01%) +77k (+ 0.02%) 338,105k 338,154k p=0.005 n=6
Parse Time 2.90s (± 0.59%) 2.87s (± 0.78%) -0.04s (- 1.21%) 2.84s 2.89s p=0.042 n=6
Bind Time 1.10s (± 1.21%) 1.10s (± 0.57%) ~ 1.09s 1.11s p=0.720 n=6
Check Time 8.07s (± 0.51%) 8.07s (± 0.18%) ~ 8.05s 8.09s p=0.871 n=6
Emit Time 4.68s (± 0.46%) 4.69s (± 0.46%) ~ 4.66s 4.71s p=1.000 n=6
Total Time 16.76s (± 0.39%) 16.73s (± 0.21%) ~ 16.68s 16.77s p=0.521 n=6
TFS - node (v14.15.1, x64)
Memory used 294,253k (± 0.00%) 294,314k (± 0.00%) +62k (+ 0.02%) 294,307k 294,327k p=0.005 n=6
Parse Time 2.37s (± 0.78%) 2.37s (± 0.64%) ~ 2.35s 2.39s p=0.451 n=6
Bind Time 1.06s (± 0.49%) 1.07s (± 0.51%) ~ 1.06s 1.07s p=0.640 n=6
Check Time 7.46s (± 0.67%) 7.49s (± 0.63%) ~ 7.41s 7.54s p=0.296 n=6
Emit Time 4.30s (± 0.56%) 4.28s (± 0.70%) ~ 4.24s 4.31s p=0.292 n=6
Total Time 15.20s (± 0.44%) 15.19s (± 0.43%) ~ 15.10s 15.28s p=0.872 n=6
material-ui - node (v14.15.1, x64)
Memory used 471,344k (± 0.00%) 471,285k (± 0.01%) -60k (- 0.01%) 471,255k 471,351k p=0.020 n=6
Parse Time 3.49s (± 0.54%) 3.44s (± 0.43%) -0.05s (- 1.48%) 3.42s 3.46s p=0.005 n=6
Bind Time 1.00s (± 0.63%) 1.00s (± 0.54%) ~ 1.00s 1.01s p=0.201 n=6
Check Time 19.00s (± 0.64%) 18.87s (± 0.70%) ~ 18.76s 19.13s p=0.078 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 23.49s (± 0.50%) 23.32s (± 0.57%) ~ 23.19s 23.57s p=0.054 n=6
xstate - node (v14.15.1, x64)
Memory used 534,448k (± 0.01%) 534,560k (± 0.00%) +112k (+ 0.02%) 534,528k 534,602k p=0.005 n=6
Parse Time 4.59s (± 0.48%) 4.47s (± 0.30%) -0.12s (- 2.61%) 4.46s 4.49s p=0.005 n=6
Bind Time 1.67s (± 0.63%) 1.81s (± 0.83%) +0.14s (+ 8.51%) 1.78s 1.82s p=0.005 n=6
Check Time 3.16s (± 0.95%) 3.15s (± 0.48%) ~ 3.14s 3.18s p=0.742 n=6
Emit Time 0.10s (± 0.00%) 0.09s (± 5.53%) 🟩-0.01s (- 6.67%) 0.09s 0.10s p=0.025 n=6
Total Time 9.53s (± 0.38%) 9.53s (± 0.18%) ~ 9.51s 9.55s p=0.936 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 52920 6
Baseline main 6

TSServer

Comparison Report - main..52920
Metric main 52920 Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,370ms (± 0.74%) 2,360ms (± 0.58%) ~ 2,350ms 2,387ms p=0.230 n=6
Req 2 - geterr 5,326ms (± 0.67%) 5,313ms (± 0.36%) ~ 5,292ms 5,344ms p=0.575 n=6
Req 3 - references 341ms (± 1.04%) 337ms (± 0.41%) -4ms (- 1.22%) 334ms 338ms p=0.007 n=6
Req 4 - navto 284ms (± 0.58%) 282ms (± 0.55%) ~ 280ms 284ms p=0.145 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 71ms (± 0.89%) 71ms (± 1.91%) ~ 69ms 73ms p=0.438 n=6
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,462ms (± 0.62%) 2,507ms (± 0.97%) +45ms (+ 1.83%) 2,470ms 2,533ms p=0.013 n=6
Req 2 - geterr 4,002ms (± 0.67%) 3,978ms (± 0.63%) ~ 3,936ms 4,001ms p=0.471 n=6
Req 3 - references 352ms (± 0.71%) 352ms (± 0.72%) ~ 349ms 355ms p=0.808 n=6
Req 4 - navto 292ms (± 0.59%) 292ms (± 0.61%) ~ 290ms 295ms p=0.868 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 68ms (± 0.60%) 69ms (± 3.40%) ~ 66ms 73ms p=0.446 n=6
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 3,164ms (± 0.64%) 3,182ms (± 0.59%) ~ 3,156ms 3,212ms p=0.294 n=6
Req 2 - geterr 1,613ms (± 0.86%) 1,616ms (± 1.21%) ~ 1,587ms 1,647ms p=0.936 n=6
Req 3 - references 103ms (± 1.13%) 103ms (± 0.79%) ~ 102ms 104ms p=0.498 n=6
Req 4 - navto 361ms (± 0.64%) 359ms (± 0.98%) ~ 352ms 361ms p=0.170 n=6
Req 5 - completionInfo count 3,136 (± 0.00%) 3,136 (± 0.00%) ~ 3,136 3,136 p=1.000 n=6
Req 5 - completionInfo 431ms (± 1.14%) 428ms (± 1.04%) ~ 421ms 433ms p=0.294 n=6
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,526ms (± 0.57%) 2,478ms (± 1.09%) -49ms (- 1.93%) 2,446ms 2,509ms p=0.013 n=6
Req 2 - geterr 5,689ms (± 0.52%) 5,675ms (± 0.55%) ~ 5,625ms 5,715ms p=0.471 n=6
Req 3 - references 350ms (± 0.50%) 350ms (± 0.67%) ~ 347ms 353ms p=0.625 n=6
Req 4 - navto 280ms (± 1.03%) 280ms (± 0.95%) ~ 278ms 285ms p=0.870 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 82ms (± 5.05%) 86ms (± 5.35%) ~ 83ms 95ms p=0.304 n=6
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,648ms (± 0.58%) 2,682ms (± 0.57%) +34ms (+ 1.28%) 2,659ms 2,705ms p=0.008 n=6
Req 2 - geterr 4,348ms (± 0.46%) 4,344ms (± 0.39%) ~ 4,316ms 4,369ms p=0.688 n=6
Req 3 - references 364ms (± 1.23%) 365ms (± 0.47%) ~ 362ms 367ms p=0.935 n=6
Req 4 - navto 287ms (± 1.17%) 284ms (± 0.90%) ~ 281ms 287ms p=0.260 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 73ms (± 1.03%) 73ms (± 1.03%) ~ 72ms 74ms p=0.487 n=6
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 3,326ms (± 0.29%) 3,317ms (± 0.24%) ~ 3,306ms 3,329ms p=0.172 n=6
Req 2 - geterr 1,830ms (± 5.45%) 1,775ms (± 0.89%) ~ 1,755ms 1,794ms p=0.810 n=6
Req 3 - references 111ms (± 0.93%) 112ms (± 2.26%) ~ 108ms 114ms p=0.463 n=6
Req 4 - navto 344ms (± 0.98%) 348ms (± 0.57%) +4ms (+ 1.16%) 346ms 351ms p=0.044 n=6
Req 5 - completionInfo count 3,136 (± 0.00%) 3,136 (± 0.00%) ~ 3,136 3,136 p=1.000 n=6
Req 5 - completionInfo 434ms (± 0.31%) 437ms (± 0.76%) ~ 434ms 443ms p=0.081 n=6
Compiler-UnionsTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,584ms (± 0.55%) 2,537ms (± 0.31%) -47ms (- 1.81%) 2,532ms 2,553ms p=0.005 n=6
Req 2 - geterr 6,024ms (± 0.45%) 5,995ms (± 0.59%) ~ 5,937ms 6,029ms p=0.148 n=6
Req 3 - references 366ms (± 0.89%) 365ms (± 0.32%) ~ 363ms 366ms p=0.331 n=6
Req 4 - navto 277ms (± 0.90%) 275ms (± 0.61%) ~ 273ms 278ms p=0.072 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 100ms (± 2.53%) 100ms (± 0.63%) ~ 99ms 101ms p=0.236 n=6
CompilerTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,802ms (± 0.55%) 2,796ms (± 0.53%) ~ 2,784ms 2,825ms p=0.471 n=6
Req 2 - geterr 4,533ms (± 2.66%) 4,394ms (± 0.31%) 🟩-139ms (- 3.07%) 4,372ms 4,412ms p=0.008 n=6
Req 3 - references 380ms (± 1.06%) 377ms (± 0.26%) ~ 376ms 378ms p=0.210 n=6
Req 4 - navto 294ms (± 1.23%) 295ms (± 0.46%) ~ 294ms 297ms p=0.572 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 87ms (± 8.85%) 91ms (±10.11%) ~ 84ms 103ms p=0.101 n=6
xstateTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 3,652ms (± 0.45%) 3,634ms (± 0.35%) ~ 3,616ms 3,649ms p=0.065 n=6
Req 2 - geterr 1,841ms (± 1.32%) 1,846ms (± 0.89%) ~ 1,827ms 1,871ms p=0.748 n=6
Req 3 - references 129ms (± 1.89%) 127ms (± 4.14%) ~ 123ms 137ms p=0.261 n=6
Req 4 - navto 374ms (± 1.17%) 373ms (± 1.30%) ~ 364ms 378ms p=0.574 n=6
Req 5 - completionInfo count 3,136 (± 0.00%) 3,136 (± 0.00%) ~ 3,136 3,136 p=1.000 n=6
Req 5 - completionInfo 455ms (± 3.45%) 453ms (± 2.17%) ~ 446ms 472ms p=0.809 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.15.1, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.15.1, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 52920 6
Baseline main 6

Startup

Comparison Report - main..52920
Metric main 52920 Delta Best Worst p-value
tsc-startup - node (v16.17.1, x64)
Execution time 142.27ms (± 0.20%) 142.59ms (± 0.20%) +0.32ms (+ 0.22%) 141.80ms 148.16ms p=0.000 n=600
tsserver-startup - node (v16.17.1, x64)
Execution time 226.62ms (± 0.18%) 227.05ms (± 0.28%) +0.43ms (+ 0.19%) 225.67ms 236.50ms p=0.000 n=600
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 228.36ms (± 0.16%) 232.06ms (± 0.37%) +3.70ms (+ 1.62%) 228.08ms 238.35ms p=0.000 n=600
typescript-startup - node (v16.17.1, x64)
Execution time 209.30ms (± 0.18%) 210.09ms (± 0.23%) +0.79ms (+ 0.38%) 208.86ms 216.53ms p=0.000 n=600
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 52920 6
Baseline main 6

Developer Information:

Download Benchmark

@jakebailey
Copy link
Member

Seems to be a percent or so? Still, pretty cool.

@DanielRosenwasser
Copy link
Member Author

My guess is that an engine would need to still check each time in case these are possibly undefined, and still needs to actually go through another function. So it won't win back the perf hit in the parser from #35282, but it gets some of it by removing the repeated indirection of accessing factory methods from the object.

@DanielRosenwasser DanielRosenwasser marked this pull request as ready for review February 22, 2023 23:49
@rbuckton
Copy link
Member

All of the factory.createX calls in parser are monomorphic (though I need to check to be sure), so this change doesn't seem to have much of an impact on total emit time, or services time. Because the new benchmark output just shows ~ for total time, I can only assume that means the change wasn't significant? If a parser improvement were going to effect services, would that not show up in the new tsserver benchmarks, or do those not adequately cover a case where this impact might be seen?

I'm a little concerned we'd be introducing complexity for little actual benefit, as reaching for new factory methods in parser would necessitate additional destructuring.

We also might want to limit the local references to factory functions to only those involved in hot paths, i.e., createToken, createIdentifier, createBinaryExpression, createParameter, createVariableDeclaration, etc. since the infrequently accessed factory functions won't give us the same benefit.

@DanielRosenwasser
Copy link
Member Author

I'm definitely open to it. Plus, I think that long-term, we should actually run a different experiment - use the functions directly, and just write a lint rule to say "nobody apart from the parser and the factory should import the create/update functions directly.

@jakebailey
Copy link
Member

jakebailey commented Feb 23, 2023

Because the new benchmark output just shows ~ for total time, I can only assume that means the change wasn't significant?

The ~ means "not significant enough to say for sure 95% of the time", to make the other numbers stand out. I think this one is just very small, so only a few parser changes appear to be significant. The other ones do appear to have a very small decrease, but with a higher p value than 0.05, so there's less confidence in them.

If a parser improvement were going to effect services, would that not show up in the new tsserver benchmarks, or do those not adequately cover a case where this impact might be seen?

They might show a difference given they do open projects like the compiler or xstate, but because the benchmarks capture the entire "open" call or the entire "geterr" call, every stage shows up together, so it doesn't move the needle very much.

@rbuckton
Copy link
Member

I'm definitely open to it. Plus, I think that long-term, we should actually run a different experiment - use the functions directly, and just write a lint rule to say "nobody apart from the parser and the factory should import the create/update functions directly.

You can't just "use the functions directly" with the current design, as they depend on closed-over parameters to control behavior like automatic parenthesis wrapping. Also, I'm working on a completely different refactor related to dropping objectAllocator and moving the various methods we add to Node in services to compiler, and I'd like to finish up that investigation before we make any changes to factory itself.

@jakebailey
Copy link
Member

I'm mostly "pro" this change, but, if we want to get this into 5.0, should we limit its scope down to the common ones? Or should we just skip this for 5.0?

@DanielRosenwasser

This comment was marked as outdated.

@typescript-bot

This comment was marked as outdated.

@DanielRosenwasser DanielRosenwasser changed the title Destructure all factory methods in the parser. Destructure common factory methods in the parser. Feb 24, 2023
@DanielRosenwasser

This comment was marked as outdated.

@typescript-bot

This comment was marked as outdated.

@DanielRosenwasser
Copy link
Member Author

@typescript-bot perf test faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Feb 24, 2023

Heya @DanielRosenwasser, I've started to run the abridged perf test suite on this PR at 1015e3a. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..52920

Metric main 52920 Delta Best Worst p-value
Angular - node (v16.17.1, x64)
Memory used 358,876k (± 0.00%) 358,864k (± 0.00%) ~ 358,841k 358,891k p=0.261 n=6
Parse Time 3.72s (± 0.47%) 3.72s (± 0.31%) ~ 3.70s 3.73s p=0.680 n=6
Bind Time 1.19s (± 0.63%) 1.19s (± 0.34%) ~ 1.19s 1.20s p=1.000 n=6
Check Time 9.46s (± 0.50%) 9.39s (± 0.51%) -0.07s (- 0.69%) 9.33s 9.43s p=0.029 n=6
Emit Time 7.92s (± 0.71%) 7.88s (± 0.19%) -0.05s (- 0.63%) 7.85s 7.89s p=0.044 n=6
Total Time 22.30s (± 0.43%) 22.18s (± 0.26%) ~ 22.09s 22.23s p=0.065 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 191,866k (± 0.68%) 191,361k (± 0.03%) ~ 191,322k 191,428k p=0.630 n=6
Parse Time 1.56s (± 1.27%) 1.56s (± 1.17%) ~ 1.54s 1.58s p=0.622 n=6
Bind Time 0.82s (± 0.00%) 0.82s (± 0.00%) ~ 0.82s 0.82s p=1.000 n=6
Check Time 10.11s (± 0.63%) 10.04s (± 0.19%) -0.07s (- 0.74%) 10.01s 10.06s p=0.030 n=6
Emit Time 3.03s (± 0.86%) 3.00s (± 0.74%) ~ 2.97s 3.03s p=0.052 n=6
Total Time 15.52s (± 0.45%) 15.41s (± 0.22%) -0.11s (- 0.69%) 15.36s 15.45s p=0.012 n=6
Monaco - node (v16.17.1, x64)
Memory used 343,079k (± 0.01%) 343,117k (± 0.00%) +38k (+ 0.01%) 343,090k 343,141k p=0.020 n=6
Parse Time 2.80s (± 0.58%) 2.79s (± 0.29%) ~ 2.78s 2.80s p=0.869 n=6
Bind Time 1.08s (± 0.38%) 1.08s (± 0.59%) ~ 1.07s 1.09s p=0.673 n=6
Check Time 7.67s (± 0.46%) 7.68s (± 0.21%) ~ 7.67s 7.71s p=0.360 n=6
Emit Time 4.43s (± 0.59%) 4.41s (± 0.45%) ~ 4.38s 4.44s p=0.329 n=6
Total Time 15.98s (± 0.21%) 15.97s (± 0.14%) ~ 15.94s 16.00s p=0.372 n=6
TFS - node (v16.17.1, x64)
Memory used 299,222k (± 0.01%) 299,267k (± 0.01%) +45k (+ 0.02%) 299,251k 299,294k p=0.005 n=6
Parse Time 2.19s (± 0.88%) 2.15s (± 0.59%) -0.04s (- 1.90%) 2.13s 2.17s p=0.007 n=6
Bind Time 1.24s (± 1.29%) 1.24s (± 1.02%) ~ 1.22s 1.26s p=1.000 n=6
Check Time 7.18s (± 0.68%) 7.18s (± 0.52%) ~ 7.14s 7.23s p=0.871 n=6
Emit Time 4.33s (± 0.66%) 4.32s (± 0.95%) ~ 4.28s 4.39s p=0.570 n=6
Total Time 14.95s (± 0.40%) 14.89s (± 0.46%) ~ 14.81s 14.96s p=0.377 n=6
material-ui - node (v16.17.1, x64)
Memory used 475,747k (± 0.01%) 475,707k (± 0.01%) ~ 475,632k 475,776k p=0.230 n=6
Parse Time 3.33s (± 0.25%) 3.29s (± 0.17%) -0.04s (- 1.25%) 3.28s 3.29s p=0.004 n=6
Bind Time 0.96s (± 0.54%) 0.96s (± 0.42%) ~ 0.96s 0.97s p=0.114 n=6
Check Time 18.07s (± 0.66%) 18.07s (± 0.50%) ~ 17.95s 18.18s p=1.000 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.36s (± 0.49%) 22.32s (± 0.39%) ~ 22.20s 22.42s p=0.748 n=6
xstate - node (v16.17.1, x64)
Memory used 545,799k (± 0.02%) 545,786k (± 0.02%) ~ 545,660k 545,931k p=0.936 n=6
Parse Time 4.25s (± 0.35%) 4.28s (± 0.32%) +0.03s (+ 0.67%) 4.26s 4.30s p=0.015 n=6
Bind Time 1.75s (± 0.86%) 1.76s (± 0.23%) +0.01s (+ 0.86%) 1.76s 1.77s p=0.044 n=6
Check Time 2.98s (± 0.51%) 2.99s (± 0.56%) ~ 2.97s 3.01s p=0.623 n=6
Emit Time 0.09s (± 4.45%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=0.405 n=6
Total Time 9.07s (± 0.29%) 9.12s (± 0.23%) +0.05s (+ 0.53%) 9.10s 9.15s p=0.020 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current 52920 6
Baseline main 6

Developer Information:

Download Benchmark

@DanielRosenwasser
Copy link
Member Author

But the results for half of these make no sense.

@jakebailey
Copy link
Member

@typescript-bot perf test faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Feb 24, 2023

Heya @jakebailey, I've started to run the abridged perf test suite on this PR at 1015e3a. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..52920

Metric main 52920 Delta Best Worst p-value
Angular - node (v16.17.1, x64)
Memory used 359,035k (± 0.01%) 359,075k (± 0.00%) +40k (+ 0.01%) 359,064k 359,088k p=0.005 n=6
Parse Time 3.73s (± 0.80%) 3.72s (± 0.14%) ~ 3.71s 3.72s p=0.560 n=6
Bind Time 1.19s (± 0.63%) 1.19s (± 0.53%) ~ 1.18s 1.20s p=0.718 n=6
Check Time 9.45s (± 0.64%) 9.41s (± 0.51%) ~ 9.36s 9.49s p=0.227 n=6
Emit Time 7.93s (± 0.62%) 7.94s (± 1.21%) ~ 7.85s 8.12s p=0.809 n=6
Total Time 22.31s (± 0.52%) 22.26s (± 0.60%) ~ 22.16s 22.51s p=0.373 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 191,374k (± 0.04%) 191,845k (± 0.69%) ~ 191,202k 194,534k p=0.471 n=6
Parse Time 1.59s (± 1.31%) 1.57s (± 0.74%) ~ 1.56s 1.59s p=0.137 n=6
Bind Time 0.83s (± 0.49%) 0.82s (± 0.77%) -0.01s (- 1.01%) 0.81s 0.83s p=0.033 n=6
Check Time 10.09s (± 0.27%) 10.07s (± 0.54%) ~ 10.00s 10.15s p=0.747 n=6
Emit Time 3.04s (± 0.77%) 3.00s (± 1.70%) ~ 2.93s 3.09s p=0.076 n=6
Total Time 15.54s (± 0.24%) 15.46s (± 0.46%) ~ 15.36s 15.57s p=0.092 n=6
Monaco - node (v16.17.1, x64)
Memory used 343,078k (± 0.00%) 343,121k (± 0.01%) +42k (+ 0.01%) 343,095k 343,156k p=0.008 n=6
Parse Time 2.81s (± 0.52%) 2.79s (± 0.70%) ~ 2.76s 2.81s p=0.062 n=6
Bind Time 1.09s (± 0.58%) 1.08s (± 0.59%) -0.01s (- 0.92%) 1.07s 1.09s p=0.031 n=6
Check Time 7.69s (± 0.67%) 7.68s (± 0.32%) ~ 7.66s 7.73s p=0.810 n=6
Emit Time 4.45s (± 1.41%) 4.40s (± 0.47%) ~ 4.38s 4.44s p=0.146 n=6
Total Time 16.04s (± 0.63%) 15.96s (± 0.26%) ~ 15.91s 16.03s p=0.126 n=6
TFS - node (v16.17.1, x64)
Memory used 299,217k (± 0.01%) 299,250k (± 0.01%) +34k (+ 0.01%) 299,230k 299,273k p=0.031 n=6
Parse Time 2.20s (± 0.85%) 2.17s (± 0.64%) -0.03s (- 1.44%) 2.15s 2.18s p=0.011 n=6
Bind Time 1.24s (± 1.10%) 1.24s (± 0.41%) ~ 1.24s 1.25s p=0.541 n=6
Check Time 7.17s (± 0.58%) 7.18s (± 0.42%) ~ 7.16s 7.24s p=0.293 n=6
Emit Time 4.34s (± 0.70%) 4.34s (± 1.02%) ~ 4.30s 4.42s p=0.745 n=6
Total Time 14.94s (± 0.52%) 14.92s (± 0.34%) ~ 14.87s 15.01s p=0.936 n=6
material-ui - node (v16.17.1, x64)
Memory used 475,712k (± 0.01%) 475,646k (± 0.01%) ~ 475,610k 475,708k p=0.066 n=6
Parse Time 3.33s (± 0.41%) 3.30s (± 0.25%) -0.04s (- 1.10%) 3.29s 3.31s p=0.004 n=6
Bind Time 0.96s (± 0.54%) 0.96s (± 0.57%) -0.01s (- 0.87%) 0.95s 0.96s p=0.038 n=6
Check Time 18.11s (± 0.47%) 18.04s (± 0.27%) ~ 17.99s 18.11s p=0.148 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.41s (± 0.37%) 22.30s (± 0.20%) -0.11s (- 0.49%) 22.25s 22.35s p=0.016 n=6
xstate - node (v16.17.1, x64)
Memory used 545,722k (± 0.01%) 545,878k (± 0.02%) +156k (+ 0.03%) 545,689k 546,038k p=0.031 n=6
Parse Time 4.27s (± 0.21%) 4.27s (± 0.45%) ~ 4.25s 4.30s p=0.935 n=6
Bind Time 1.75s (± 0.72%) 1.76s (± 0.46%) ~ 1.74s 1.76s p=0.337 n=6
Check Time 2.99s (± 0.51%) 2.99s (± 0.33%) ~ 2.98s 3.00s p=0.446 n=6
Emit Time 0.09s (± 5.53%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=0.174 n=6
Total Time 9.11s (± 0.28%) 9.11s (± 0.31%) ~ 9.08s 9.15s p=1.000 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current 52920 6
Baseline main 6

Developer Information:

Download Benchmark

@DanielRosenwasser
Copy link
Member Author

That one seems like the benchmark that I can actually pretend to trust!

Do we want to remove the "chain" functions? I feel like those are rarer.

@jakebailey
Copy link
Member

I'm fine either way. Without doing all of them the, difference feels really small already, but this can't hurt.

@DanielRosenwasser
Copy link
Member Author

On a serious note, we've got two runs that show slightly faster parsing results on a few projects - so that does feel like an improvement.

The biggest suspicion to me is the xstate benchmark, which is dominated by parse time at the moment, but whose parse time doesn't get faster at all outside of Node 14. It feels like there are some savings to be had there

@DanielRosenwasser DanielRosenwasser merged commit 3f7bf69 into main Feb 24, 2023
@DanielRosenwasser DanielRosenwasser deleted the destructTheFactory branch February 24, 2023 20:52
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants