Skip to content

[JSC] Settle an Evaluate() of an EVALUATING module when the walk pops its cycle - #871

Draft
robobun wants to merge 1 commit into
robobun/f48be7fb/module-finishes-inside-walkfrom
robobun/ff435ce1/import-of-evaluating-module
Draft

robobun wants to merge 1 commit into
robobun/f48be7fb/module-finishes-inside-walkfrom
robobun/ff435ce1/import-of-evaluating-module

Conversation

@robobun

@robobun robobun commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #832. Notes say why.

Problem

  • A job can run while a module is evaluated: a microtask checkpoint, or a require() of an ES module. When it continues an import() of a module that is still on the stack of that evaluation, the promise never settles. A debug build fails ASSERTION FAILED: status() == Status::Unlinked || status() == Status::Linked || status() == Status::EvaluatingAsync || status() == Status::Evaluated (CyclicModuleRecord.cpp(545)).
  • CyclicModuleRecord::evaluate() stores a capability on the EVALUATING record (CyclicModuleRecord.cpp:609 on main). Nothing resolves it.

Fix

  • evaluate() of an EVALUATING record returns a promise that VM::m_moduleEvaluationWaiters keeps, and writes nothing to the record.
  • When the record leaves the stack, the promise becomes the capability of the cycle root, or follows the one that the root has.
  • Verified: JSTests/stress/import-evaluating-module-in-job-inside-evaluation*.js (31 graphs, main gets 23 wrong) and JSTests/modules.
  • Self-reviewed: 29 concerns raised, 27 addressed (Notes).

Background

Downsides

  • Per VM +8 B. Per module record, and per import() in promises and jobs: +0. Instructions per popped cycle: not measured yet (Notes).
  • A module that awaits such an import() of a cycle member that waits for it now deadlocks, as without a job. Bun 1.4.3 completes it.
  • An unhandled import() of a failing module now rejects: exit code 1, as in Node.js. Bun 1.4.3 exits with 0.
Notes

The program and the releases. Four files: entry.mjs imports a.mjs and then calls import("./a.mjs"). a.mjs imports b.mjs. b.mjs calls createRequire(import.meta.url)("./r.mjs"). r.mjs calls import("./a.mjs").

Node.js 26.3.0 Bun 1.3.13 Bun 1.3.14 Bun 1.4.2 and main Bun with this change
the import() in r.mjs fulfilled fulfilled pending pending fulfilled
the import() in entry.mjs fulfilled fulfilled pending pending fulfilled
both, with drainMicrotasks() of bun:jsc in b.mjs and no require() fulfilled pending pending fulfilled

A top-level await of that later import() hangs the process on Bun 1.4.2. No user has reported any of this.

The review of the loader rewrite saw the assertion with drainMicrotasks() and left it (WebKit#57827, 2026-04-02): it "occurs only if microtasks are explicitly drained while a microtask is already being processed", which "wouldn't ever occur on the web", and the fix there was "a linear scan during dynamic imports". Bun has such calls. This change adds one status compare to an import(), and no scan.

The calls in Bun that run a job while a module is evaluated. Each leaves the import() pending on Bun 1.4.2, and each fulfils it with this change:

  • drainMicrotasks() of bun:jsc
  • a require() of an ES module, also as import.meta.require() in the output of bun build --splitting --target=bun
  • an async HTMLRewriter handler, in transform() of a string
  • Bun.build() with a plugin whose setup() is async
  • expect(promise).resolves and .rejects, expect(asyncFn).toThrow(), an async expect.extend matcher
  • a require() of a file that calls an async macro

The graphs of the tests. Each is a directory under JSTests/stress/resources/import-evaluating-module-in-job-inside-evaluation/. A body calls import() for a module on the stack and then drainMicrotasks(). The test reads the state of each import inside the walk, after it, and after one more import() of the same module. "the import" is the one that a body started inside the walk.

graph main, release shell this change
dependency, job-later-in-the-walk, cycle-members, collected-while-waiting the import and each later import of the module stay pending fulfilled when the walk has finished the module or its cycle. A body that runs later in the same walk sees it fulfilled
cycle-with-the-module-of-the-evaluate, async-cycle-members, async-cycle-with-the-module-of-the-evaluate the import of a cycle member stays pending fulfilled with the cycle
async-cycle-member fulfilled when the async body of the member ends, before the root of the cycle has run fulfilled after the root
awaited-by-later-module a later module awaits the import: the evaluation never ends it ends
body-throws, dependency-throws, cycle-root-body-throws, import-started-before-the-failure the import stays pending rejected with the error of the evaluation
sibling-throws-later pending fulfilled: the module had finished before the evaluation failed
nested-evaluation, nested-evaluation-fails (a second Evaluate() inside the walk, through a deferred namespace) pending the inner evaluation settles the imports of the modules on its own stack only
deferred-namespace-of-failed-module the deferred namespace of a module whose evaluation threw reads its binding it throws that error
awaited-by-dependency-of-cycle-root completes the deadlock of the Downsides
finishes-on-the-stack, imported-after-it-finished-on-the-stack, rejects-on-the-stack, rejects-on-the-stack-then-body-throws, imported-after-a-job-rejected-it wrong, and wrong in another way with #832 alone (below) right

The other eight give the same on main: module-of-the-evaluate, top-level-await, waits-for-async-dependency, top-level-await-rejects, awaited-by-dependency, importer-of-module-on-the-stack, import-defer (a debug shell of main asserts in link()) and terminated (the watchdog ends the walk while an import waits).

Why this is on top of #832. #832 is for a job that finishes or rejects a module inside the walk. Its review asked to settle the promise of such an import() "when the walk pops the cycle", and #832 left that to a change for the import(). Five graphs need both changes. For finishes-on-the-stack: main fulfils the import before the cycle has run and never finishes the root. #832 alone finishes the root and leaves the import pending. Both together fulfil it after the root.

Random graphs. 7,800 graphs of 2 to 6 modules with cycles, top-level awaits (of a settled value, or of a promise that the test resolves later), throws, and bodies that call import() for modules of the graph and then drainMicrotasks(). A checker imports each module once more after the evaluation. Against #832 alone: the bodies run in the same order in all 7,800, and no later import changes. #832 alone leaves an import pending, or settles it differently from a later import, in 3,133 graphs. This change does in none. A debug shell ran 4,500 of them with no failed assertion. oven-sh/bun has the generator in its pull request for this (test/js/bun/resolve/import-evaluating-fuzz.ts), with seeds in a test.

Which test fails without which line. WebKit CI runs JSTests without assertions. On a release shell, each of these makes graphs fail by value when it is taken out: the branch in evaluate(), the call before the pop, the call after the pop, the call in step 9.a, the loop over the members of the cycle, the pipeFrom branch (cycle-members, async-cycle-members, async-cycle-with-the-module-of-the-evaluate), each arm of settleAdoptedEvaluationWaiter(), the lookup in evaluationWaiter() (dependency), the change in ModuleRegistryEntry::error() (imported-after-a-job-rejected-it) and the loop in VM::visitAggregateImpl (collected-while-waiting). With the two checks of #832 put back, no result changes.

Cost.

  • sizeof(VM) 109,184 to 109,192 (one HashMap). sizeof(JSModuleRecord) is 384 before and after.
  • JSPromise::create calls and microtask queue entries per import(), from gdb breakpoints on debug shells (20 and 60 imports): 8 and 12 for an evaluated module, 15 and 16 for a new one-module graph, before and after.
  • An Evaluate() of a record on a stack makes one promise, as main does. A record whose cycle root has a capability already costs one more job (pipeFrom).
  • A cycle of 20,000 modules that each import themselves inside the walk, then 20,000 modules that the walk pops one at a time: 0.84 to 0.87 s (3 runs, release build without LTO), and 0.58 to 0.62 s without the second 20,000. main takes 0.93 s and 0.62 s and leaves the 20,000 imports pending. A pop looks up the records that it pops, and only while a promise waits.
  • Not measured yet: instructions per popped cycle and per Evaluate(), and the sizes of the functions. They need the release build of this pull request, to compare with the one of [JSC] A job that finishes a module from inside a module body does not strand or start its importers early #832.

Self-review. Not done: an issue for this bug (this account cannot open one), and landing oven-sh/bun#41997 before this (it is a separate pull request).

Not changed.

@robobun

robobun commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

Status

  • Reproduced on Bun 1.3.14, 1.4.2 and main with the four-file program of the Notes (createRequire() of an ES module that calls import()): both imports stay pending. Bun 1.3.13 and Node.js 26.3.0 fulfil them.
  • Reproduced in the jsc shell of main with JSTests/stress/import-evaluating-module-in-job-inside-evaluation.js: a release shell gets 23 of the 31 graphs wrong, and a debug shell fails the assertion in link().
  • The Bun pull request with the tests follows when this pull request has its preview build.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Preview build of 09fb39b: autobuild-preview-pr-871-09fb39bf

@robobun
robobun force-pushed the robobun/f48be7fb/module-finishes-inside-walk branch from e441007 to ab1eb44 Compare October 22, 2026 01:09
… its cycle

ECMA-262 has no job that runs while a module graph is evaluated. A host can
run one from a module body: a microtask checkpoint, or a synchronous
require() of an ES module. The continuation of an import() is such a job,
and it calls Link() and Evaluate() for the module that it imports. That
module can be on the stack of the InnerModuleEvaluation that the body runs
in: its status is EVALUATING.

Evaluate() asserted that the status is not EVALUATING. In a release build
it made a new capability, stored it in [[TopLevelCapability]] of the
record, and returned it: the walk of an EVALUATING record returns at once.
Nothing resolves the capability of a record that another Evaluate()
finishes, so the import() never settled, and step 4 gave the same promise
to each later Evaluate() of the module.

Evaluate() of an EVALUATING record now returns a promise that
VM::m_moduleEvaluationWaiters keeps, and writes nothing to the record. The
record of the outer Evaluate() has a capability, and step 4 still returns
that one. Link() accepts the status: InnerModuleLinking already returns at
once for it.

The Evaluate() that has the record on its stack takes the promise out when
the record leaves the stack. Before step 16 of InnerModuleEvaluation pops
a cycle, the promises of its records go to the root. The first one becomes
the capability of the root, as for an Evaluate() that is called right
after the pop, and the others follow that capability. After the pop a root
that is EVALUATED fulfils it. AsyncModuleExecutionFulfilled or
AsyncModuleExecutionRejected settles it for a root that is
EVALUATING-ASYNC. Step 9.a of Evaluate() rejects the promise of each
record that it gives the error to.

This change is on top of the one for a job that finishes a module inside
the walk. No record on a stack has a capability from a job now, so the two
checks for that in asyncExecutionFulfilled() go. A record that a job
rejected stays EVALUATING there, and step 9.a can give it another error.
ModuleRegistryEntry::error() does not answer an import() of such a record
with the first error: the import() waits for the evaluation.

A graph with no such job pays one check of an empty map for each cycle
that the walk pops, and one status compare in each Evaluate() that walks.
@robobun

robobun commented Oct 31, 2026

Copy link
Copy Markdown
Collaborator Author

Cross-reference: #961 changes the end of AOT::Modules::evaluateWithOwnStack, which stands in for the end of CyclicModuleRecord::evaluate() where compiled code evaluates a module without a record. It settles the capability that evaluate() stores in the record of an EVALUATING root today.

With this change evaluate() of an EVALUATING record writes nothing to the record and keeps the promise in VM::m_moduleEvaluationWaiters. A root that the AOT run time evaluates never is on the stack of innerModuleEvaluation of runtime/, so its waiters need a settle at that same place in aot/runtime/AOTModuleEvaluation.cpp. The programs of oven-sh/bun#47538 fail if they stay pending.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant