Make TypeScript itself ESM-only, made possible by require(ESM) #58419

jakebailey · 2024-05-02T22:59:53Z

nodejs/node#51977 (behind --experimental-require-module in Node 22) enables Node to require ESM so long as that ESM does not make use of top-level await. TypeScript would only need top-level await to construct ts.sys at startup, which must be defined when the environment is detected to be Node. Via nodejs/node#52599 and nodejs/node#51977, we can synchronously access Node's built-ins, and therefore can offer up a TLA-free public API, enabling TypeScript to ship as ESM-only without breaking CJS users, once require(ESM) is unflagged.

The bulk of this change is just the extension rewriting, which we may want to consider taking to main just to make a PR like this one clearer.

For casual onlookers, I don't think this PR is going to "happen" any time soon (where "now" is currently May 2024); require(ESM) is new, flagged, and TS will likely need to support versions of Node until they're sufficiently EOL for us to feel comfortable dropping them. But it's incredible that this is possible.

TODO:

Fix public API dts
Figure out __esModule
- The real question is the context of require(ESM) having __esModule. But I suspect that exporting default fixes this.
Consider looking for require if process.getBuiltinModule is not found
Export maps? Actually make the package importable? (does this work?)
Maybe still ship CJS behind a condition to get this out sooner?
- Dual package hazard is very scary here. Maybe acceptable with module condition/top-level prop?
Stop bundling? Share code between tsc.js/typescript.js?
Probably a billion other things
Is this even a good idea before we figure out a better public API than our current huge omnibus?
@types/diff is typed wrong

typescript-bot · 2024-05-02T23:00:08Z

Thanks for the PR! It looks like you've changed the TSServer protocol in some way. Please ensure that any changes here don't break consumers of the current TSServer API. For some extra review, we'll ping @sheetalkamat, @mjbvz, @zkat, and @joj for you. Feel free to loop in other consumers/maintainers if necessary.

jakebailey · 2024-05-07T01:53:08Z

This has been rebased now that we're using extensions on main, condensing this down to a pretty small change (though there's still stuff not working).

typescript-bot · 2024-05-07T03:55:47Z

Looks like you're introducing a change to the public API surface area. If this includes breaking changes, please document them on our wiki's API Breaking Changes page.

Also, please make sure @DanielRosenwasser and @RyanCavanaugh are aware of the changes, just as a heads up.

jakebailey · 2024-05-07T04:41:23Z

There are still a bunch of TODOs, but this is actually green! Wow!

jakebailey · 2024-05-31T22:01:16Z

nodejs/node#52762 was merged a bit ago; I should be able to retarget this PR at nightly.

jakebailey · 2024-10-17T20:46:35Z

@typescript-bot perf test this faster [email protected]

typescript-bot · 2024-10-17T20:46:38Z

Starting jobs; this comment will be updated as builds start and complete.

Command	Status	Results
`perf test this faster [email protected]`	✅ Started	👀 Results

typescript-bot · 2024-10-17T21:24:22Z

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Compiler-Unions - node (v23.0.0, x64)}
_Errors	₃₁	₃₁	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_62,340	_62,340	_~	_~	_~	_{p=1.000 n=6}
_Types	_50,379	_50,379	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{193,727k (± 0.02%)}	_{194,442k (± 0.02%)}	_{+715k (+ 0.37%)}	_194,412k	_194,500k	_{p=0.005 n=6}
_{Parse Time}	_{1.14s (± 1.99%)}	_{1.17s (± 1.21%)}	_{+0.03s (+ 3.08%)}	_1.15s	_1.19s	_{p=0.023 n=6}
_{Bind Time}	_0.56s	_{0.58s (± 0.71%)}	_{+0.02s (+ 3.27%)}	_0.57s	_0.58s	_{p=0.002 n=6}
_{Check Time}	_{7.49s (± 0.36%)}	_{7.60s (± 0.81%)}	_{+0.11s (+ 1.42%)}	_7.53s	_7.70s	_{p=0.008 n=6}
_{Emit Time}	_{2.28s (± 0.33%)}	_{2.40s (± 0.53%)}	_{🔻+0.12s (+ 5.19%)}	_2.39s	_2.42s	_{p=0.004 n=6}
_{Total Time}	_{11.47s (± 0.34%)}	_{11.74s (± 0.41%)}	_{+0.27s (+ 2.38%)}	_11.69s	_11.81s	_{p=0.005 n=6}
_{angular-1 - node (v23.0.0, x64)}
_Errors	₃₃	₃₃	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_947,886	_947,886	_~	_~	_~	_{p=1.000 n=6}
_Types	_410,840	_410,840	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{1,225,465k (± 0.00%)}	_{1,226,690k (± 0.00%)}	_{+1,225k (+ 0.10%)}	_1,226,609k	_1,226,739k	_{p=0.005 n=6}
_{Parse Time}	_{5.65s (± 3.19%)}	_{5.70s (± 3.64%)}	_~	_5.39s	_5.85s	_{p=0.228 n=6}
_{Bind Time}	_{1.74s (±11.31%)}	_{1.83s (±10.71%)}	_{🔻+0.10s (+ 5.56%)}	_1.71s	_2.14s	_{p=0.046 n=6}
_{Check Time}	_{25.87s (± 0.50%)}	_{26.35s (± 0.35%)}	_{+0.48s (+ 1.85%)}	_26.23s	_26.49s	_{p=0.005 n=6}
_{Emit Time}	_{13.00s (± 0.66%)}	_{13.68s (± 0.55%)}	_{🔻+0.68s (+ 5.22%)}	_13.63s	_13.83s	_{p=0.005 n=6}
_{Total Time}	_{46.26s (± 0.45%)}	_{47.57s (± 0.27%)}	_{+1.31s (+ 2.84%)}	_47.42s	_47.76s	_{p=0.005 n=6}
_{mui-docs - node (v23.0.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_2,530,445	_2,530,445	_~	_~	_~	_{p=1.000 n=6}
_Types	_916,594	_916,594	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{2,340,106k (± 0.00%)}	_{2,341,324k (± 0.00%)}	_{+1,218k (+ 0.05%)}	_2,341,198k	_2,341,420k	_{p=0.005 n=6}
_{Parse Time}	_{9.33s (± 0.77%)}	_{9.55s (± 0.55%)}	_{+0.22s (+ 2.30%)}	_9.47s	_9.62s	_{p=0.005 n=6}
_{Bind Time}	_{2.23s (± 0.54%)}	_{2.30s (± 0.59%)}	_{+0.07s (+ 3.14%)}	_2.28s	_2.32s	_{p=0.004 n=6}
_{Check Time}	_{74.99s (± 1.12%)}	_{75.36s (± 0.73%)}	_~	_74.86s	_76.39s	_{p=0.229 n=6}
_{Emit Time}	_{0.38s (±48.10%)}	_{0.30s (± 1.35%)}	_~	_0.30s	_0.31s	_{p=0.248 n=6}
_{Total Time}	_{86.93s (± 1.01%)}	_{87.51s (± 0.59%)}	_{+0.58s (+ 0.67%)}	_87.04s	_88.46s	_{p=0.045 n=6}
_{self-build-src - node (v23.0.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_1,258,032	_1,258,086	_{+54 (+ 0.00%)}	_~	_~	_{p=0.001 n=6}
_Types	_266,259	_266,404	_{+145 (+ 0.05%)}	_~	_~	_{p=0.001 n=6}
_{Memory used}	_{2,422,147k (± 0.01%)}	_{2,433,135k (± 0.01%)}	_{+10,988k (+ 0.45%)}	_2,432,859k	_2,433,402k	_{p=0.005 n=6}
_{Parse Time}	_{4.08s (± 0.89%)}	_{4.27s (± 0.65%)}	_{🔻+0.19s (+ 4.65%)}	_4.23s	_4.31s	_{p=0.005 n=6}
_{Bind Time}	_{1.75s (± 0.60%)}	_{1.80s (± 0.29%)}	_{+0.05s (+ 2.75%)}	_1.80s	_1.81s	_{p=0.004 n=6}
_{Check Time}	_{26.72s (± 0.23%)}	_{26.97s (± 0.49%)}	_{+0.25s (+ 0.93%)}	_26.87s	_27.21s	_{p=0.005 n=6}
_{Emit Time}	_{1.97s (± 0.75%)}	_{2.02s (± 0.49%)}	_{+0.05s (+ 2.71%)}	_2.01s	_2.04s	_{p=0.005 n=6}
_{Total Time}	_{34.56s (± 0.26%)}	_{35.07s (± 0.40%)}	_{+0.51s (+ 1.48%)}	_34.92s	_35.32s	_{p=0.005 n=6}
_{self-build-src-public-api - node (v23.0.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_1,258,032	_1,258,086	_{+54 (+ 0.00%)}	_~	_~	_{p=0.001 n=6}
_Types	_266,259	_266,404	_{+145 (+ 0.05%)}	_~	_~	_{p=0.001 n=6}
_{Memory used}	_{2,503,622k (± 0.01%)}	_{2,526,649k (± 0.02%)}	_{+23,026k (+ 0.92%)}	_2,526,334k	_2,527,559k	_{p=0.005 n=6}
_{Parse Time}	_{4.34s (± 1.03%)}	_{4.47s (± 0.72%)}	_{+0.13s (+ 3.11%)}	_4.43s	_4.51s	_{p=0.005 n=6}
_{Bind Time}	_{1.51s (± 0.27%)}	_{1.56s (± 0.33%)}	_{+0.04s (+ 2.98%)}	_1.55s	_1.56s	_{p=0.003 n=6}
_{Check Time}	_{27.55s (± 0.31%)}	_{27.89s (± 0.25%)}	_{+0.35s (+ 1.25%)}	_27.82s	_27.98s	_{p=0.005 n=6}
_{Emit Time}	_{1.98s (± 0.87%)}	_{2.08s (± 0.66%)}	_{🔻+0.10s (+ 5.21%)}	_2.06s	_2.10s	_{p=0.005 n=6}
_{Total Time}	_{35.38s (± 0.32%)}	_{36.01s (± 0.23%)}	_{+0.63s (+ 1.78%)}	_35.94s	_36.15s	_{p=0.005 n=6}
_{self-compiler - node (v23.0.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_261,789	_261,815	_{+26 (+ 0.01%)}	_~	_~	_{p=0.001 n=6}
_Types	_106,511	_106,599	_{+88 (+ 0.08%)}	_~	_~	_{p=0.001 n=6}
_{Memory used}	_{439,547k (± 0.02%)}	_{440,625k (± 0.03%)}	_{+1,078k (+ 0.25%)}	_440,483k	_440,879k	_{p=0.005 n=6}
_{Parse Time}	_{3.09s (± 1.50%)}	_{3.19s (± 1.11%)}	_{+0.10s (+ 3.18%)}	_3.15s	_3.22s	_{p=0.005 n=6}
_{Bind Time}	_{1.13s (± 1.22%)}	_{1.16s (± 1.15%)}	_{+0.03s (+ 2.96%)}	_1.14s	_1.17s	_{p=0.007 n=6}
_{Check Time}	_{14.80s (± 0.35%)}	_{15.09s (± 0.15%)}	_{+0.29s (+ 1.94%)}	_15.05s	_15.11s	_{p=0.005 n=6}
_{Emit Time}	_{1.15s (± 0.65%)}	_{1.20s (± 1.01%)}	_{+0.04s (+ 3.91%)}	_1.18s	_1.21s	_{p=0.005 n=6}
_{Total Time}	_{20.18s (± 0.27%)}	_{20.63s (± 0.22%)}	_{+0.46s (+ 2.27%)}	_20.58s	_20.71s	_{p=0.005 n=6}
_{ts-pre-modules - node (v23.0.0, x64)}
_Errors	₆₈	₆₈	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_225,919	_225,919	_~	_~	_~	_{p=1.000 n=6}
_Types	_94,415	_94,415	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{372,382k (± 0.01%)}	_{373,221k (± 0.01%)}	_{+839k (+ 0.23%)}	_373,153k	_373,283k	_{p=0.005 n=6}
_{Parse Time}	_{2.50s (± 1.06%)}	_{2.58s (± 2.04%)}	_{+0.08s (+ 3.13%)}	_2.51s	_2.65s	_{p=0.015 n=6}
_{Bind Time}	_{1.40s (± 1.05%)}	_{1.43s (± 0.73%)}	_{+0.03s (+ 2.38%)}	_1.42s	_1.45s	_{p=0.012 n=6}
_{Check Time}	_{12.60s (± 0.52%)}	_{12.85s (± 0.50%)}	_{+0.24s (+ 1.93%)}	_12.80s	_12.97s	_{p=0.005 n=6}
_{Emit Time}	_0.00s	_0.00s	_~	_~	_~	_{p=1.000 n=6}
_{Total Time}	_{16.52s (± 0.50%)}	_{16.87s (± 0.49%)}	_{+0.35s (+ 2.13%)}	_16.76s	_17.00s	_{p=0.005 n=6}
_{vscode - node (v23.0.0, x64)}
_Errors	₃	₃	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_3,118,670	_3,118,670	_~	_~	_~	_{p=1.000 n=6}
_Types	_1,074,955	_1,074,955	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{3,211,514k (± 0.02%)}	_{3,212,413k (± 0.00%)}	_{+899k (+ 0.03%)}	_3,212,340k	_3,212,534k	_{p=0.005 n=6}
_{Parse Time}	_{12.42s (± 1.14%)}	_{12.73s (± 0.52%)}	_{+0.31s (+ 2.51%)}	_12.64s	_12.83s	_{p=0.005 n=6}
_{Bind Time}	_{4.02s (± 3.00%)}	_{4.16s (± 2.99%)}	_~	_4.03s	_4.28s	_{p=0.065 n=6}
_{Check Time}	_{70.77s (± 0.49%)}	_{72.30s (± 0.36%)}	_{+1.53s (+ 2.16%)}	_71.94s	_72.70s	_{p=0.005 n=6}
_{Emit Time}	_{22.72s (±10.89%)}	_{25.78s (± 0.86%)}	_{🔻+3.05s (+13.43%)}	_25.52s	_26.13s	_{p=0.005 n=6}
_{Total Time}	_{109.93s (± 2.00%)}	_{114.97s (± 0.35%)}	_{🔻+5.04s (+ 4.58%)}	_114.39s	_115.37s	_{p=0.005 n=6}
_{webpack - node (v23.0.0, x64)}
_Errors	₀	₀	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_286,241	_286,241	_~	_~	_~	_{p=1.000 n=6}
_Types	_116,005	_116,005	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{437,913k (± 0.02%)}	_{438,988k (± 0.03%)}	_{+1,075k (+ 0.25%)}	_438,769k	_439,069k	_{p=0.005 n=6}
_{Parse Time}	_{4.18s (± 0.47%)}	_{4.24s (± 0.67%)}	_{+0.06s (+ 1.52%)}	_4.19s	_4.27s	_{p=0.009 n=6}
_{Bind Time}	_{1.66s (± 1.25%)}	_{1.73s (± 1.34%)}	_{🔻+0.07s (+ 4.33%)}	_1.70s	_1.76s	_{p=0.005 n=6}
_{Check Time}	_{16.35s (± 0.32%)}	_{16.75s (± 0.07%)}	_{+0.40s (+ 2.45%)}	_16.74s	_16.77s	_{p=0.005 n=6}
_{Emit Time}	_0.00s	_0.00s	_~	_~	_~	_{p=1.000 n=6}
_{Total Time}	_{22.19s (± 0.30%)}	_{22.72s (± 0.21%)}	_{+0.54s (+ 2.43%)}	_22.67s	_22.80s	_{p=0.005 n=6}
_{xstate-main - node (v23.0.0, x64)}
_Errors	₃	₃	_~	_~	_~	_{p=1.000 n=6}
_Symbols	_543,130	_543,130	_~	_~	_~	_{p=1.000 n=6}
_Types	_181,889	_181,889	_~	_~	_~	_{p=1.000 n=6}
_{Memory used}	_{486,951k (± 0.01%)}	_{487,800k (± 0.01%)}	_{+849k (+ 0.17%)}	_487,769k	_487,844k	_{p=0.005 n=6}
_{Parse Time}	_{3.46s (± 1.35%)}	_{3.55s (± 0.96%)}	_{+0.09s (+ 2.75%)}	_3.51s	_3.60s	_{p=0.005 n=6}
_{Bind Time}	_{1.11s (± 0.57%)}	_{1.16s (± 1.52%)}	_{🔻+0.04s (+ 4.05%)}	_1.13s	_1.18s	_{p=0.004 n=6}
_{Check Time}	_{17.81s (± 0.40%)}	_{18.09s (± 0.32%)}	_{+0.28s (+ 1.54%)}	_18.02s	_18.17s	_{p=0.005 n=6}
_{Emit Time}	_0.00s	_0.00s	_~	_~	_~	_{p=1.000 n=6}
_{Total Time}	_{22.38s (± 0.52%)}	_{22.79s (± 0.28%)}	_{+0.41s (+ 1.85%)}	_22.69s	_22.89s	_{p=0.005 n=6}

System info unknown

Hosts

node (v23.0.0, x64)

Scenarios

Compiler-Unions - node (v23.0.0, x64)
angular-1 - node (v23.0.0, x64)
mui-docs - node (v23.0.0, x64)
self-build-src - node (v23.0.0, x64)
self-build-src-public-api - node (v23.0.0, x64)
self-compiler - node (v23.0.0, x64)
ts-pre-modules - node (v23.0.0, x64)
vscode - node (v23.0.0, x64)
webpack - node (v23.0.0, x64)
xstate-main - node (v23.0.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

Developer Information:

Download Benchmarks

jakebailey · 2024-10-17T21:25:49Z

Huh, where is the perf hit coming from here...?

typescript-bot assigned jakebailey May 2, 2024

typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels May 2, 2024

jakebailey force-pushed the its-esm branch from 0290acc to 69aa00f Compare May 2, 2024 23:30

This was referenced May 2, 2024

process: add process.getBuiltinModule(id) nodejs/node#52762

Closed

Use explicit extensions for imports within src #58421

Merged

jakebailey force-pushed the its-esm branch 6 times, most recently from 9a1860f to 42acb3b Compare May 7, 2024 01:41

jakebailey force-pushed the its-esm branch 2 times, most recently from bed73cc to 739c9bf Compare May 7, 2024 02:42

jakebailey mentioned this pull request May 9, 2024

Load unit tests more consistently (pulled from ESM branch) #58481

Merged

jakebailey force-pushed the its-esm branch 10 times, most recently from 18d1d83 to 423f584 Compare May 10, 2024 04:25

jakebailey force-pushed the its-esm branch from 423f584 to 84d3641 Compare May 31, 2024 22:08

This was referenced Jun 10, 2024

Module resolution: ESM-style namespace importing is broken #58817

Closed

ts object has no properties when loading typescript.js in a browser with v5.5-rc #58893

Closed

jakebailey force-pushed the its-esm branch from 2b28c0b to 64ac0d2 Compare June 25, 2024 17:29

jakebailey mentioned this pull request Jun 25, 2024

Use regular imports instead of require where possible #59017

Merged

jakebailey force-pushed the its-esm branch 4 times, most recently from 7c70c64 to 208e064 Compare June 28, 2024 17:32

jakebailey force-pushed the its-esm branch from 212ec1a to f3cb303 Compare July 24, 2024 21:49

jakebailey force-pushed the its-esm branch 2 times, most recently from 793228e to 4c232d5 Compare August 14, 2024 16:36

jakebailey force-pushed the its-esm branch from 4c232d5 to 520f2be Compare September 17, 2024 21:30

jakebailey added 2 commits October 17, 2024 12:50

Mark as type=module

7294e47

Update .gulp.js

f6853e7

jakebailey force-pushed the its-esm branch from 520f2be to e272d8f Compare October 17, 2024 20:30

jakebailey added 3 commits October 17, 2024 13:39

Make things work in ESM with getBuiltinModule

aa792d9

Enable CI

d6cb557

Make module format errors red

8a63274

jakebailey force-pushed the its-esm branch from e272d8f to 8a63274 Compare October 17, 2024 20:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make TypeScript itself ESM-only, made possible by require(ESM) #58419

Make TypeScript itself ESM-only, made possible by require(ESM) #58419

jakebailey commented May 2, 2024 •

edited

Loading

typescript-bot commented May 2, 2024

jakebailey commented May 7, 2024

typescript-bot commented May 7, 2024

jakebailey commented May 7, 2024

jakebailey commented May 31, 2024

jakebailey commented Oct 17, 2024

typescript-bot commented Oct 17, 2024 •

edited

Loading

typescript-bot commented Oct 17, 2024

tsc

jakebailey commented Oct 17, 2024

Make TypeScript itself ESM-only, made possible by require(ESM) #58419

Are you sure you want to change the base?

Make TypeScript itself ESM-only, made possible by require(ESM) #58419

Conversation

jakebailey commented May 2, 2024 • edited Loading

typescript-bot commented May 2, 2024

jakebailey commented May 7, 2024

typescript-bot commented May 7, 2024

jakebailey commented May 7, 2024

jakebailey commented May 31, 2024

jakebailey commented Oct 17, 2024

typescript-bot commented Oct 17, 2024 • edited Loading

typescript-bot commented Oct 17, 2024

tsc

jakebailey commented Oct 17, 2024

jakebailey commented May 2, 2024 •

edited

Loading

typescript-bot commented Oct 17, 2024 •

edited

Loading