require(esm): a macro whose file is already in the graph no longer spins, and a shared module is fetched once (WebKit bump for oven-sh/WebKit#675) - #42915
Conversation
…les that require(esm) has already fetched
A synchronous module load uses a fetch that the host already delivered, and import() beneath it does not wait for another load's promise. The tag is a preview build of the open PR. It has to move to the autobuild-<sha> of the merge commit before this lands.
|
Status
|
|
Updated 5:28 AM PT - Sep 16th, 2026
❌ @robobun, your commit 6142e89 has 1 failures in
Add 🧪 To try this PR locally: bunx bun-pr 42915That installs a local version of the PR into your bun-42915 --bun |
||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Answers to the review:
The next push also moves the pin to a new preview build: oven-sh/WebKit#675 got one more commit from its review (a top-level load of a just-fetched module built the module record twice). |
…ommonJS module builds it once
oven-sh/WebKit#675 got one more commit: a top-level load of a module that the enclosing require(esm) has just fetched continues from the host's fetch promise, so the module record is built once. The size check compares with the latest build of main, which pins a newer WebKit (c28156899e, built with LLVM 23). This preview is 65513e295c plus the fix, because the base branch is still on the LLVM 21 toolchain. That difference puts bun-freebsd-aarch64 1.6 MB over, and it goes away when the pin moves to the merged commit.
|
This review has no findings to act on. The four threads from the first review have replies and are resolved. What remains before merge is in the first line of the PR body: oven-sh/WebKit#675 has to land, and then the pin moves to its |
Problem
require()of an ES module still spins when a module in its graph calls a macro and the graph already holds the macro's file (the twotest.todocases of macros: do not spin when a macro runs beneath require() of an ES module #42778). Bun 1.3.13 runs these.await import(<macro file>), andMacro::initwaits for it. What settles the file's fetch or load promise is parked in therequire()'sVM::m_synchronousModuleQueue, which cannot drain before the macro returns.require()of a just-fetched sibling throwsrequire() async module ... is unsupported.Fix
require(esm)does not change. A dynamic import beneath a synchronous load does not wait for another load's promise.todoIf(isDebug || isASAN)test.macro-test.test.tsandrequire-esm-fetched-once.test.tsfail. All pass with the pin. Self-reviewed: the engine change was rebuilt twice around the findings (see Notes).Background
require()d ES module that is on the main thread, inside the loader's fetch hook.m_synchronousModuleQueueis Bun's addition to JSC forrequire(esm). The loader's promise reactions go there, and only thatrequire()drains it.Notes
WEBKIT_VERSIONisautobuild-preview-pr-675-d953c3bd:65513e295cplus oven-sh/WebKit#675.mainhas moved to a newer WebKit since (c28156899e, built with LLVM 23). The preview stays on65513e295cwhile the base branch is on the LLVM 21 toolchain, and both move when #42778 is rebased. The binary size check compares withmain, so it reportsbun-freebsd-aarch641.6 MB over for that WebKit difference alone. The pin commit carries[skip size check]for that reason.What each test needs (debug build with ASAN, base = this branch without the new pin):
ASSERTION FAILED: module->loadedModules().size() <= loadedModulesCountBefore + 1ReferenceError: two is not defined2 21 1require-esm-fetched-once: a module that two siblings import is loaded onceonLoadsees it twicerequire-esm-fetched-once:require()of an ES module that the enclosing graph has just fetchedrequire() async module "leaf.mjs" is unsupportedrequire-esm-fetched-once:import()of a CommonJS module that the enclosing graph has just fetchedThe three tests in
test/js/bun/resolve/require-esm-fetched-once.test.tsdo not use a macro and do not need #42778. They fail onmain(1.4.3) and pass on 1.3.13. If the pin should land before #42778, say so and I split it out.The macro-mode transpile. A macro's module is transpiled with
is_macro_runtime: macro calls in it are not expanded. On the base, a second request for a module that therequire()had already transpiled fetched it again. From a macro's module that second transpile was the macro-mode one, and it replaced the source that the program runs: a module with its own macro call then failed withReferenceError. With the new pin the loader uses the source that is already there.Still open (one
test.todo, and see oven-sh/WebKit#675):require()inside a macro of a module that the outerrequire()has started to load reports it as an async module. Arequire()is a graph load of its own, and it links on the module's load promise, which the outerrequire()'s parked reactions fulfill. 1.3.13 runs this.import()spins. That completion goes to the regular event loop, which the macro's wait does not run.require-esm-fetched-oncetests still need the engine change.#33180 is not touched. A first version of the engine change fetched again from the top-level
loadModule, which also maderequire()of a module that is in flight on another thread work. oven-sh/WebKit#662 does exactly that, so it is not here. The tests of #37185 still fail with this pin.Self-review. A first review found the overlap with oven-sh/WebKit#662 and #396 and the macro-mode transpile problem, and the engine change was rebuilt around them. A second review of an intermediate version found that dropping the
pipeFromjob changed the job order of plainrequire(esm)(false async-module errors, a changed CommonJS evaluation order). The final version keeps the job order. A differential run of 2,800 random module graphs against the base shows no graph that is worse and three that are fixed.Suites run on the fixed engine (local build, see oven-sh/WebKit#675 for how): all of
test/bundler/transpiler/macro-test.test.ts,test/js/bun/resolve/,test/js/node/module/,test/js/bun/plugin/plugins.test.ts,test/js/bun/test/mock/,test/cli/run/require-cache.test.ts(without the RSS leak tests),test/regression/issue/24387.test.ts,30493.test.ts.load the same empty JS file 2000 timestimes out on this machine on the base too.[policy-decision:webkit] gate passed · iteration 0 · 3 files touched
passes on PR (with fix)
diff hotspot
gate history · 2 passed · 0 rejected · iteration 0
evidence per changed file