Skip to content

bun test: fail the test or hook whose done() received an error - #39112

Open
robobun wants to merge 6 commits into
mainfrom
farm/c44e7648/done-err-attribution
Open

robobun wants to merge 6 commits into
mainfrom
farm/c44e7648/done-err-attribution

Conversation

@robobun

@robobun robobun commented Aug 15, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • A test that reports its failure with done(new Error(...)) is printed as (pass) when it runs in a concurrent group (it.concurrent, describe.concurrent, or any test under bun test --concurrent). The error is printed as # Unhandled error between tests instead, and the summary reads e.g. 4 pass, 0 fail, 4 errors. The exit code is still 1, but the JUnit report has no <failure> for the test, and a -t rerun of the test looks green.
  • A beforeAll / beforeEach / afterEach / afterAll hook calling done(err) behaves the same way even serially: the hook is not reported as failed, the dependent tests still run and pass (a failing beforeEach / beforeAll does not even skip the bodies), the error is counted as unhandled. A hook that throws fails them.
  • it.concurrent.failing("x", done => done(err)) is reported as a failure (the error is not credited to the test, so it "passed").
  • node:test reports every failure by calling bun:test's done callback, so under bun test --concurrent every failing node:test test is reported as passing, and a suite-level before() / after() hook that throws (reported through a beforeAll / afterAll done callback) is reported that way even serially.
  • Cause: bun_test_done_callback (src/runtime/test_runner/bun_test.rs) reports the error through the generic VirtualMachine::uncaught_exception, which attributes it with BunTest::get_current_state_data(). For a group with more than one sequence that returns entry_data: None, and for a hook jest::on_unhandled_rejection demotes it to RefDataValue::Start; either way Execution::handle_uncaught_exception returns ShowUnhandledErrorBetweenTests and never marks the sequence failed, so the add_result a few lines later completes it as a pass. Serially in a test body it worked only because get_current_state_data() happened to resolve to the same test.

Fix

  • DoneCallback now records, at creation, the file (Weak) and the RefDataValue that run_test_callback was invoked with; done(err) calls on_uncaught_exception keyed by that value, which is exactly how a throw or rejection from the same callback is reported (run_test_callback / bun_test_then_or_catch). The result for a test body, a hook, test.failing and JUnit is therefore identical to throwing from that callback, in serial and concurrent groups alike.
  • This is correct for every moment done can fire: synchronously in the body or in its microtask drain (before the completion ref exists), later from a macrotask (ref stamped), or after the entry already finished. In the last case get_current_and_valid_execution_sequence rejects the stale value and the error is reported as Unhandled error between tests, like a late rejection already is; previously such a late done(err) failed (and completed) whichever test happened to be running.
  • Retry attempts of the same test are the one case the stamp cannot tell apart until bun test: ignore completions from an earlier attempt of a retried test #38876 lands (it adds a per-attempt generation to this same check): a late done(err) from a timed-out attempt is still charged to the attempt running at the time and triggers another retry, as it already is serially on main. In a concurrent group of several tests main happened to report it as an unhandled error (exit 1) and this PR retries there too (exit 0); with bun test: ignore completions from an earlier attempt of a retried test #38876 both are rejected as stale. The retry: 2 fixture posted on test runner: fail dependent tests when a hook's done() receives an error #33089 belongs to whichever of the two lands second.
  • Completion is unchanged: the error is reported first, then the existing ref / add_result / run_next_tick path runs exactly once. A side effect is that the next test now starts on the next tick after done(err), as it already did after done(); before, the generic path advanced the runner synchronously inside the done() call, which is why two stack frames from the previous test's done() call disappear from the existing snapshot in test-error-code-done-callback.test.ts.
  • node:test's runtime t.skip() / t.todo() mark (js_node_test_mark_result) reads the same stamp, which replaces Execution.on_stack_entry_data (added for exactly this "done fired before the ref was stamped" case); its existing serial and --concurrent tests still pass.
  • Not changed: done(false) / done("") still count as errors, and a second done() call is still a no-op (bun:test: fail the test when done(err) is called after done() #34041 is about that; it touches the same lines and would report through the same owner stamp). An async callback that also declares done still completes as soon as its promise fulfills (the Fulfilled arm of run_test_callback), so a done(err) it fires later is one of the stale cases above: reported as an unhandled error instead of, as before, failing the next test. Whether such a callback should wait for done is a separate question. test runner: fail dependent tests when a hook's done() receives an error #33089 (closed in favor of this PR) attributed done(err) through the ref as well but fell back to get_current_state_data() when the ref is not stamped yet, so the synchronous / microtask concurrent cases above (and node:test under --concurrent) stayed broken there; its hook-kind and node:test suite-hook cases are carried here.
  • Verified:
    • test/js/bun/test/test-error-code-done-callback.test.ts: one fixture (sync, microtask and macrotask done(err), done(), it.failing, done(err) from each of beforeAll / beforeEach / afterEach / afterAll) must produce the same per-test results serially, under --concurrent, and with it.concurrent / describe.concurrent; plus three stale-done(err) cases (timed out, body threw after scheduling it on a macrotask / microtask) that must not fail the test running at the time. All 7 fail on the current release (USE_SYSTEM_BUN=1), pass with bun bd test.
    • test/js/junit-reporter/junit.test.js: a concurrent group's done(err) failures land in their own <testcase> (before: failures="0", no <failure> elements).
    • test/js/node/test_runner/node-test.test.ts: the failing-hooks fixture reports 10 fail under --concurrent too (before: all ten reported as passing, ten unhandled errors).
    • test/js/node/test_runner/node-test.test.ts + fixtures/30-failing-suite-hooks.js: two suites whose before() / after() throw report 1 pass, 2 fail and the body under the failed before() does not run (before: 2 pass, 0 fail, 2 errors, body ran).
    • Also green locally on the debug build: the rest of test/js/bun/test/ that covers done callbacks, hooks, failing/retry/skip and concurrency, test/cli/test/bun-test.test.ts, the rest of junit.test.js and node-test.test.ts, and the vendored test/js/node/test/parallel/test-runner-*.js files run the way scripts/runner.node.mjs runs them.

Background

  • Execution model: after collection, a file's tests and hooks are laid out as ExecutionSequences (one test plus its beforeEach/afterEach entries), grouped into ConcurrentGroups. A serial test is a group of one sequence; consecutive concurrent tests share one group and run interleaved.
  • RefDataValue: the runner's handle for "this completion belongs to group G, sequence S, entry E, repeat N". run_test_callback is invoked with one; the promise .then/.catch and the done callback hand it back via add_result, and Execution::get_current_and_valid_execution_sequence refuses it once the sequence has moved on to another entry or completed, which is what makes late completions harmless (telling a retried attempt from its predecessor is bun test: ignore completions from an earlier attempt of a retried test #38876).
  • on_uncaught_exception(value, &RefDataValue) marks that sequence failed (or passed for test.failing, skipping the rest of the sequence for a hook) and prints the error as that entry's failure, also recording it for JUnit. With a value it cannot resolve it prints Unhandled error between tests and bumps the error counter instead.
  • get_current_state_data() is the fallback used for genuinely stray errors (an uncaught exception from a timer, an unhandled rejection): it can only point at the single sequence of a serial group, which is why it is the wrong tool for an error whose owner is known.
  • The done callback's existing r#ref is a refcounted copy of the same RefDataValue, but it is only created after the callback returns (so a promise-returning callback and done() can agree on who completes the test); a done(err) called from the body, from the microtask drain, or after the body threw never sees it, hence the separate stamp taken at creation.
Notes

Rebase notes: main moved twice under this branch. The two conflicts and their resolutions:


[review] gate passed · iteration 2 · 8 files touched

fails on main (without fix)
ASAN without fix: 10 FAILED
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/bun/test/test-error-code-done-callback.test.ts test/js/junit-reporter/junit.test.js test/js/node/test_runner/node-test.test.ts
bun test v1.4.1 (861e9ae04)

test/js/junit-reporter/junit.test.js:
(pass) junit reporter > should generate valid junit xml for passing tests %s [789.46ms]
(pass) junit reporter > should generate valid junit xml for passing tests %s [439.81ms]
[String: "/tmp/junit-comprehensive_BvrVf2"]
(pass) junit reporter > more scenarios [1046.39ms]
(pass) junit reporter > should report only the final result for a retried test [349.21ms]
(pass) junit reporter > produces well-formed XML when test names contain control characters [349.78ms]
(pass) junit reporter > escapes the classname attribute exactly once [354.33ms]
(pass) junit reporter > keeps the test body's error in <failure> when afterEach also throws [331.60ms]
(pass) junit reporter > includes the error type, message and stack in <failure> [331.60ms]
(pass) junit reporter > prints a stack frame whose source is not a file path as-is in <failure> [443.18ms]
632 
... (truncated)

release without fix: 10 FAILED
bun test v1.4.1-canary.1 (861e9ae04)

test/js/junit-reporter/junit.test.js:
(pass) junit reporter > should generate valid junit xml for passing tests %s [15.45ms]
(pass) junit reporter > should generate valid junit xml for passing tests %s [10.05ms]
[String: "/tmp/junit-comprehensive_92NwUC"]
(pass) junit reporter > more scenarios [17.60ms]
(pass) junit reporter > should report only the final result for a retried test [8.35ms]
(pass) junit reporter > produces well-formed XML when test names contain control characters [8.37ms]
(pass) junit reporter > escapes the classname attribute exactly once [7.94ms]
(pass) junit reporter > keeps the test body's error in <failure> when afterEach also throws [7.17ms]
(pass) junit reporter > includes the error type, message and stack in <failure> [7.57ms]
(pass) junit reporter > prints a stack frame whose source is not a file path as-is in <failure> [16.79ms]
632 |     const xmlContent = await file(junitPath).text();
633 |     const result = await new Promise((resolve, reject) => {
634 |       xml2js.parseString(xmlContent, { strict: true }, (err, r) => (err ? reject(err) : resolve(r)));
635 |     });
636 |     const suite = result.
... (truncated)
passes on PR (with fix)
ASAN with fix: all passed
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/bun/test/test-error-code-done-callback.test.ts test/js/junit-reporter/junit.test.js test/js/node/test_runner/node-test.test.ts
bun test v1.4.1 (861e9ae04)

test/js/junit-reporter/junit.test.js:
(pass) junit reporter > should generate valid junit xml for passing tests %s [675.40ms]
(pass) junit reporter > should generate valid junit xml for passing tests %s [489.84ms]
[String: "/tmp/junit-comprehensive_mtT1mu"]
(pass) junit reporter > more scenarios [980.97ms]
(pass) junit reporter > should report only the final result for a retried test [330.56ms]
(pass) junit reporter > produces well-formed XML when test names contain control characters [328.96ms]
(pass) junit reporter > escapes the classname attribute exactly once [324.53ms]
(pass) junit reporter > keeps the test body's error in <failure> when afterEach also throws [370.34ms]
(pass) junit reporter > includes the error type, message and stack in <failure> [395.63ms]
(pass) junit reporter > prints a stack frame whose source is not a file path as-is in <failure> [441.88ms]
(pass
... (truncated)

release with fix: all passed
$ bun scripts/build.ts --profile=release
[configured] bun-profile → bun (stripped)
  target       linux-x64-gnu
  build type   Release
  build dir    ./build/release
  revision     863d0c82b0
  features     baseline

23 deps, 129 codegen, 1172 objects in 672ms

ninja: Entering directory `/workspace/bun/build/release'
[1/1244] install /workspace/bun
bun install v1.4.1-canary.1 (861e9ae04)

Checked 26 installs across 63 packages (no changes) [4.00ms]
[2/1244] gen ErrorCode+*.h
[3/1244] install /workspace/bun/packages/bun-error
bun install v1.4.1-canary.1 (861e9ae04)

Checked 1 install across 2 packages (no changes) [1.00ms]
[4/1244] gen bindgenv2
[5/1244] install /workspace/bun/src/node-fallbacks
bun install v1.4.1-canary.1 (861e9ae04)

Checked 111 installs across 104 packages (no changes) [11.00ms]
[6/1244] gen bake.{client,server,error}.js
-> bake.client.js, bake.server.js, bake.error.js
[7/1244] fetch tinycc
[tinycc] up to date
[8/1243] fetch libjpeg-turbo
[libjpeg-turbo] up to date
[9/1216] fetch zlib
[zlib] up to date
[10/1216] gen .bind.ts → GeneratedBindings.cpp
[11/1216] esbuild bun-error

  ../../build/release/codegen/bun-error/index.js       34.9kb
  ../
... (truncated)
diff hotspot
src/runtime/test_runner/DoneCallback.rs            |  13 +-
 src/runtime/test_runner/Execution.rs               |  24 +--
 src/runtime/test_runner/bun_test.rs                |  29 ++--
 src/runtime/test_runner/jest.rs                    |  41 ++---
 .../bun/test/test-error-code-done-callback.test.ts | 169 ++++++++++++++++++++-
 test/js/junit-reporter/junit.test.js               |  39 +++++
 .../test_runner/fixtures/30-failing-suite-hooks.js |  20 +++
 test/js/node/test_runner/node-test.test.ts         |  30 ++++
 8 files changed, 299 insertions(+), 66 deletions(-)

gate history · 4 passed · 0 rejected · iteration 2

evidence per changed file
file                                                      reads  edits  tests
src/runtime/test_runner/DoneCallback.rs                       2      3      0
src/runtime/test_runner/Execution.rs                          0      0      0
src/runtime/test_runner/bun_test.rs                           2      2      0
src/runtime/test_runner/jest.rs                               2      3      0
test/js/bun/test/test-error-code-done-callback.test.ts        0      0      0
test/js/junit-reporter/junit.test.js                          2      0      0
…/js/node/test_runner/fixtures/30-failing-suite-hooks.js      0      0      0
test/js/node/test_runner/node-test.test.ts                    0      0      0

root cause · written by the author bot

The test runner attributed done(err) errors using the shared on-stack entry data at call time, which misattributed failures when the callback fired from a concurrent group, a hook, or after a retry, since the currently executing entry was not necessarily the one that created the callback. The fix stamps each DoneCallback at creation with a weak reference to its owning BunTest and the RefDataValue its entry was invoked with, so bun_test_done_callback and node:test result marking route errors to the correct sequence regardless of what is on the stack when the callback runs. If the o…

@coderabbitai

coderabbitai Bot commented Aug 15, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

  • Run on-demand review

On-demand reviews are free for the next 26 days. After that, they cost $0.25 per reviewed file.

Or wait 8 minutes for your next included review.

View limit details

Limit details: You’ve used all 5 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 6b480f72-a4b8-43ef-858f-e11844a122e4

📥 Commits

Reviewing files that changed from the base of the PR and between 82123d3 and 863d0c8.

📒 Files selected for processing (8)
  • src/runtime/test_runner/DoneCallback.rs
  • src/runtime/test_runner/Execution.rs
  • src/runtime/test_runner/bun_test.rs
  • src/runtime/test_runner/jest.rs
  • test/js/bun/test/test-error-code-done-callback.test.ts
  • test/js/junit-reporter/junit.test.js
  • test/js/node/test_runner/fixtures/30-failing-suite-hooks.js
  • test/js/node/test_runner/node-test.test.ts

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Aug 15, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: the fix is complete and verified. The new tests in test/js/bun/test/test-error-code-done-callback.test.ts, test/js/junit-reporter/junit.test.js and test/js/node/test_runner/node-test.test.ts fail without the src change and pass with it. The branch is rebased on main (see the rebase notes in the PR description).

The latest CI run (build 105890) has no red tests: every failure passed on retry and none touches the test runner. Ready for a maintainer.

@github-actions

Copy link
Copy Markdown
Contributor

This PR may be a duplicate of:

  1. test runner: fail dependent tests when a hook's done() receives an error #33089 - Same fix in the same code: reroutes done(err) out of the generic vm.uncaught_exception path into BunTest::on_uncaught_exception keyed by the callback's RefDataValue in bun_test_done_callback/DoneCallback.rs, though it leaves the concurrent and node:test cases unfixed.

🤖 Generated with Claude Code

@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

Not a duplicate of #33089, though the two overlap. #33089 was opened for done(err) in hooks; this PR is for the concurrent-group case (the bug it fixes is that it.concurrent / --concurrent / node:test failures reported through done(err) are printed as passing). Both route done(err) through on_uncaught_exception, but #33089 falls back to get_current_state_data() whenever done fires before the completion ref exists (synchronously, from the microtask drain, or after the body threw), and that fallback cannot name a sequence inside a concurrent group, so the cases this PR is about are still broken on that branch. This PR stamps the owner on the DoneCallback at creation instead, which covers the hook cases from #33089 as well (tested here with beforeEach / afterEach), the concurrent cases, and stale done(err) calls. I left a note on #33089; if it lands first this rebases to the remaining delta.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. The fix is well-reasoned — stamping the RefDataValue on the DoneCallback at creation and routing done(err) through on_uncaught_exception with that key mirrors exactly how a throw/rejection from the same callback is already attributed. Because it changes pass/fail attribution in the test runner (serial hooks, concurrent groups, test.failing, JUnit, node:test) and shifts an observable timing edge (next test now starts on the next tick after done(err), hence the two dropped snapshot frames), a human look would still be worthwhile.

What was checked:

  • on_stack_entry_data has no remaining references after removal; js_node_test_mark_result now reads the same owner stamp instead.
  • New DoneCallback fields (Weak, RefDataValue) drop via the Box in finalize; only r#ref needs the explicit .deref().
  • Stale done(err) is rejected by get_current_and_valid_execution_sequence and lands as "Unhandled error between tests" — covered by three explicit test cases.
  • The re-entrancy shape of strong.get().on_uncaught_exception(...) matches the existing bun_test_then_or_catch path; no new aliasing hazard.
Extended reasoning...

Overview

This PR fixes done(err) attribution in bun test: previously the error was routed through the generic VirtualMachine::uncaught_exception → get_current_state_data() path, which cannot name a sequence inside a concurrent group and demotes hook errors to RefDataValue::Start. The result was that done(err) in a concurrent test, or in any beforeEach/afterEach, printed as # Unhandled error between tests while the test itself was reported as (pass). The fix stamps the owning RefDataValue (and a Weak<BunTestCell>) on the DoneCallback at creation in run_test_callback, and bun_test_done_callback calls BunTest::on_uncaught_exception keyed by that stamp — the same path a throw or rejection from the callback already takes. js_node_test_mark_result reads the same stamp, which lets Execution.on_stack_entry_data be deleted. Four Rust files change (~60 lines net), plus three test files gaining ~200 lines of coverage.

Security risks

None. This is test-runner-internal error attribution; no untrusted input parsing, no auth/crypto/permissions.

Level of scrutiny

High. This is the test runner's pass/fail attribution logic — a bug here silently misreports test outcomes, which is exactly the failure mode the PR is fixing. It also touches concurrent execution (interleaved sequences, stale-completion rejection) and produces two observable behavior changes: (1) two stack frames disappear from an existing snapshot because the next test now starts on the next tick after done(err) rather than synchronously inside it, and (2) a late done(err) after its test finished is now reported as unhandled-between-tests instead of failing whichever test happens to be running. Both are improvements, but they're the kind of edge a maintainer should sign off on.

Other factors

The test coverage is thorough: one fixture asserted identical across serial / --concurrent / it.concurrent+describe.concurrent, covering sync/microtask/macrotask done(err), it.failing, and beforeEach/afterEach; three stale-done(err) cases; a JUnit test for concurrent done(err); and node:test's failing-hooks fixture under --concurrent. I verified on_stack_entry_data has no remaining references, that the new DoneCallback fields drop correctly via the Box in finalize (only r#ref needs the explicit .deref() because RefPtr has no Drop), and that the strong.get().on_uncaught_exception(...) re-entrancy shape matches the existing bun_test_then_or_catch path. No CODEOWNERS apply. Deferring because pass/fail attribution changes in the runner core warrant a maintainer's eyes even when the mechanism is sound.

@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

Nothing to change from that review. The two behavior edges it points at (the next test starting on the following tick after done(err), and a stale done(err) being reported as an unhandled error instead of failing the test that happens to be running) are intentional and are the ones described in the Fix section; both are pinned by the snapshot update and the three stale-done cases in test-error-code-done-callback.test.ts.

@robobun

robobun commented Aug 15, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 5:15 PM PT - Aug 25th, 2026

✅ @robobun, your commit 863d0c82b0ed2e7eb6d0d9f086f7b2b36b6f4200 passed in Build #105890! 🎉


🧪   To try this PR locally:

bunx bun-pr 39112

That installs a local version of the PR into your bun-39112 executable, so you can run:

bun-39112 --bun

@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 10:05 AM PT - Aug 15th, 2026

@robobun, your commit d4652f1 is building: #98301

Comment thread src/runtime/test_runner/DoneCallback.rs Outdated
Comment thread src/runtime/test_runner/bun_test.rs Outdated
Comment thread src/runtime/test_runner/jest.rs Outdated
Comment thread src/runtime/test_runner/DoneCallback.rs Outdated
Comment thread src/runtime/test_runner/jest.rs Outdated
@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

#33089 has been closed in favor of this PR. Its cases were checked against this branch and the ones not already pinned here were added in 1768287: beforeAll / afterAll done(err) in the fixture (so all four hook kinds are covered in the serial and both concurrent runs), and fixtures/30-failing-suite-hooks.js, a node:test file whose suite-level before() / after() throw (node:test reports those through beforeAll / afterAll done callbacks). The PR body is updated accordingly.

One case from the discussion on #33089 is worth carrying once #38876 lands: a late done(err) from a timed-out attempt of a retry test. On this branch alone it still fails the attempt that is running at the time (attempt 3 then runs and the file ends 1 pass, exit 0), because telling attempts of the same sequence apart is what #38876 adds; with both changes the owner stamped here should be rejected as stale like the other late cases.

While adding the beforeAll case: under --concurrent / describe.concurrent, a failing beforeAll also skips the concurrent tests that follow its describe (the next sibling extends the describe's last group, so failure_skip_to covers it). That reproduces with a throwing beforeAll on the release binary, so it is unrelated to this change and is being handled separately; the fixture places the beforeAll case last so the expected output does not depend on it.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. Because it reworks error attribution in the test runner's done-callback path (unsafe raw-pointer state on DoneCallback, removal of on_stack_entry_data, and an intentional behavior change to when the next test starts after done(err)), a human look from someone familiar with the Execution/RefDataValue lifecycle would still be worthwhile.

Checked: the stamped owner: RefDataValue is only passed to on_uncaught_exception after buntest_weak.upgrade() succeeds, and get_current_and_valid_execution_sequence compares the stored entry pointer without dereferencing it, so a stale owner is rejected rather than UAF'd. called is now set via mem::replace before on_uncaught_exception runs user JS, so a re-entrant done() is a no-op (previously it could re-report). The removed on_stack_entry_data had js_node_test_mark_result as its only reader; that now reads the same stamp.

Extended reasoning...

Overview

This PR fixes done(err) attribution in bun test: previously a done(new Error(...)) in a concurrent group, or in any hook even serially, was routed through the generic VirtualMachine::uncaught_exception path and printed as # Unhandled error between tests while the test/hook itself was reported as passing. The fix stamps each DoneCallback at creation with a Weak<BunTest> and the RefDataValue that run_test_callback was invoked with, then bun_test_done_callback calls BunTest::on_uncaught_exception keyed by that value — the same path a throw or promise rejection from the same callback already takes. js_node_test_mark_result (node:test's runtime t.skip()/t.todo()) is rewired to read the same stamp, which lets Execution.on_stack_entry_data be deleted. Four Rust files change (~80 net lines) plus ~250 lines of new tests across four test files.

Security risks

None. This is test-runner-internal error routing; no untrusted input parsing, auth, crypto, or network surface is touched.

Level of scrutiny

Medium-high. The change is small and the mechanism is clearly the right layer (attribute the error to the entry the done was created for, exactly as throws/rejections already are), but it lives in the test runner's execution core where lifetime, aliasing, and re-entrancy invariants are hand-managed through raw pointers and UnsafeCell. It also carries two intentional behavior changes: the next test starts on the next tick after done(err) (matching done() without an error; two stack-frame lines drop from an existing snapshot as a result), and a stale done(err) — one whose entry has already completed — is now reported as an unhandled error rather than being pinned on whichever test happens to be running. Both are pinned by tests and match how late promise rejections are already handled, but a maintainer should confirm they agree with the semantics.

Other factors

  • Test coverage is thorough: one fixture asserts identical per-test results serially, under --concurrent, and with it.concurrent/describe.concurrent across sync/microtask/macrotask done(err), done(), it.failing, and all four hook kinds; three stale-done(err) cases pin the "don't blame the running test" behavior; JUnit and node:test (--concurrent and suite-level before()/after()) get their own tests. All are stated to fail on the release build and pass on the debug build.
  • Memory/aliasing: I verified the new fields don't introduce a UAF — owner is only consumed after Weak::upgrade succeeds (arena entries alive), and the validator compares the raw entry pointer without dereferencing it. called = true is now set before on_uncaught_exception can re-enter JS, closing a (pre-existing) window where a re-entrant done() could double-report. The strong.get().on_uncaught_exception(...) pattern matches what bun_test_then_or_catch already does.
  • Dead-code removal: on_stack_entry_data was added specifically for the case this PR now covers via the stamp; its only reader is rewired here and the field/cell/scopeguard are removed together.
  • Open edge: the author notes a late done(err) from a timed-out attempt of a retry test still lands on the current attempt because attempts of the same sequence aren't yet distinguishable; that depends on #38876 and is out of scope here.
  • The comment-cop bot's long-comment flags have been addressed in the latest commits.

@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

Nothing to change from this second review either. It covers the same two behavior edges as the first (next test starts on the following tick after done(err), stale done(err) reported as an unhandled error), both intentional and pinned by tests. The retry/timeout attempt edge it mentions is out of scope here and waits on #38876, as noted in the PR body.

Comment thread src/runtime/test_runner/bun_test.rs

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The clippy clone_on_copy issue from the previous pass is fixed in 3ec364e; this run found no bugs. Given the scope — error attribution across concurrent groups, all four hook kinds, and node:test, plus the acknowledged retry-attempt edge that waits on #38876 — a maintainer look would still be worthwhile.

What was reviewed:

  • done(err) now routes through on_uncaught_exception keyed by the owner stamped at DoneCallback creation, matching the throw/reject path; the completion path (r#ref / add_result / run_next_tick) is unchanged.
  • on_stack_entry_data is fully removed (no remaining references); js_node_test_mark_result reads the same stamp.
  • buntest_weak: Weak and owner: RefDataValue (Copy) drop with the DoneCallback box in finalize — no new refcount to balance.
  • Stale done(err) after the entry finished is rejected by get_current_and_valid_execution_sequence and reported as an unhandled error, pinned by the three late-done test cases.
Extended reasoning...

Overview

The PR fixes error attribution for done(err): instead of routing through the generic VirtualMachine::uncaught_exception (which attributes via get_current_state_data() — unable to name a sequence in a concurrent group, and demoted to Start for hooks), the DoneCallback now records at creation the file (Weak) and the RefDataValue that run_test_callback was invoked with, and done(err) calls BunTest::on_uncaught_exception keyed by that value. This is the same attribution path a throw or rejection from the same callback already uses. Net native change is small (~-20 LOC after removing the on_stack_entry_data workaround it supersedes); ~250 lines of new tests cover serial, --concurrent, .concurrent, all four hook kinds, test.failing, JUnit, node:test failures under --concurrent, node:test suite-level before()/after(), and three stale-done(err) cases.

Security risks

None. This is test-runner error-attribution plumbing; no untrusted input parsing, no auth/crypto, no network.

Level of scrutiny

Moderate-to-high. The test runner is a critical path and this changes attribution across concurrent execution, hooks, and node:test integration. Two intentional behavior edges (next test starts on the following tick after done(err); a stale done(err) is now reported as an unhandled error rather than failing whichever test happens to be running) are explained in the PR body and pinned by tests. The retry-attempt edge (a late done(err) from a timed-out attempt still charged to the running attempt) is out of scope pending #38876, which adds a per-attempt generation to the same validity check.

Other factors

  • My previous inline finding (clone_on_copy on the now-Copy RefDataValue) is fixed at both sites in 3ec364e; jest.rs already used the copy form.
  • Verified on_stack_entry_data has no remaining references anywhere in src/, and DoneCallback::create_unbound has exactly one caller (run_test_callback), which passes cfg_data — the same value the promise then/catch and completion ref use.
  • Memory: the new buntest_weak (a Weak<BunTestCell>) and owner (a Copy enum of non-owning pointers) drop automatically when DoneCallback::finalize drops the Box<Self>; no explicit release needed and no new intrusive refcount is introduced.
  • All comment-cop threads and my prior thread are resolved; #33089 was closed in favor of this PR and its cases are carried here.
  • Test coverage is thorough and follows harness conventions (tempDir, bunEnv, concurrent pipe drain, test.concurrent.each for the matrix, exit-code asserted last). The PR body's evidence shows the new tests fail on main and pass with the fix on both ASAN debug and release.

@robobun

robobun commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator Author

Nothing to change from this pass. It confirms the clippy fix in 3ec364e and reports no bugs. The PR is ready for a maintainer review.

@robobun
robobun force-pushed the farm/c44e7648/done-err-attribution branch from 3ec364e to f787346 Compare August 21, 2026 02:29

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I re-reviewed after the clone_on_copy fix in f787346 and found no bugs. Because this reworks how the test runner attributes done(err) across concurrent groups, hooks, and retries — with stated interactions with #38876 and user-visible behavior edges (stale done(err) now unhandled; next-tick start after done(err)) — a maintainer look is still worthwhile.

What was reviewed:

  • The owner/buntest_weak stamp on DoneCallback and its use in bun_test_done_callback / js_node_test_mark_result — the Weak upgrade correctly falls back to vm.uncaught_exception when the file is gone.
  • on_stack_entry_data removal — grepped, no remaining references; its sole consumer now reads the stamp.
  • Confirmed both .clone() sites on the now-Copy RefDataValue are dropped (bun_test.rs:822, :1137).
Extended reasoning...

Overview

The PR stamps each DoneCallback at creation with a Weak<BunTestCell> and the RefDataValue its entry was invoked with, so done(err) (and node:test's runtime t.skip()/t.todo()) can be attributed to the correct sequence even inside a concurrent group or from a hook. It removes Execution.on_stack_entry_data (whose only consumer, js_node_test_mark_result, now reads the stamp instead), rewrites bun_test_done_callback to route the error through BunTest::on_uncaught_exception keyed by the stamped owner, and updates the existing snapshot in test-error-code-done-callback.test.ts to reflect that the next test now starts on the following tick after done(err). Test coverage is thorough: a single fixture asserted identical per-test results across serial / --concurrent / it.concurrent / describe.concurrent; all four hook kinds; it.failing; three stale-done(err) cases; a JUnit test for concurrent-group attribution; and two node:test cases (failing hooks under --concurrent, failing suite-level before()/after()).

Security risks

None. The change is confined to test-runner result attribution and reporting; no parsing of untrusted input, no auth/crypto/permissions surface.

Level of scrutiny

Moderate-to-high. The Rust changes are small (~60 net lines in src/) but sit in the test runner's completion/attribution state machine, which is dense with raw-pointer aliasing rules (UnsafeCell, NonNull into arena-owned entries) and re-entrancy hazards. The behavior changes are intentional and pinned by tests, but they interact with retry generations (#38876) and touch how a late done(err) is charged — the kind of edge a maintainer familiar with the runner should sign off on.

Other factors

  • My prior inline finding (workspace-denied clippy::clone_on_copy on the now-Copy RefDataValue) is resolved in f787346; both call sites now copy by value, matching jest.rs:557.
  • The on_stack_entry_data field and its scopeguard restore are fully removed; grep confirms no remaining references, so the deletion is complete.
  • The Weak upgrade in bun_test_done_callback correctly falls back to the generic vm.uncaught_exception when the file's BunTest is already torn down, and js_node_test_mark_result returns undefined in the same case — no dangling deref.
  • The new DoneCallback fields (BunTestPtrWeak, RefDataValue) both have proper Drop semantics (Weak drops itself; RefDataValue is Copy with only non-owning pointers), so the existing finalize needs no change.
  • The comment-cop bot's "paragraph-long comment" flags were addressed in 0877317/6a69da4; the remaining comments in the diff are one-liners.

@robobun

robobun commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator Author

Nothing to change from this pass either. It confirms the earlier findings are resolved and reports no bugs. The PR stays ready for a maintainer.

done(err) was reported through the generic uncaught-exception path, which
attributes errors with get_current_state_data(). That cannot name a sequence
inside a concurrent group and files hook errors under "between tests", so a
concurrent test (or any test under --concurrent, including every node:test
failure) and any hook calling done(err) printed "Unhandled error between
tests" and was then reported as passing.

Stamp each DoneCallback with the file and RefDataValue it was created for and
report done(err) through on_uncaught_exception keyed by that value, the same
way a throw or rejection from the callback is reported. A stale done(err) from
an entry that already finished (timed out, or threw before the runner stamped
its completion ref) fails the runner's validity check and is reported as an
unhandled error instead of being charged to whatever entry is running.

node:test's runtime t.skip()/t.todo() mark uses the same stamp, which replaces
Execution.on_stack_entry_data.
…hooks

Adds the beforeAll and afterAll hook kinds to the done(err) fixture (so
all four hook kinds are pinned in the serial and both concurrent modes)
and a node:test fixture whose suite-level before()/after() hooks throw,
which node:test reports through those hooks' done callbacks. These are
the cases #33089 was opened for; it is being closed in favor of this PR.
@robobun

robobun commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator Author

#41865 adopts the DoneCallback owner stamp from this PR for done(err) only (it needs done(err) off the generic unhandled-error hook so that test.failing keeps inverting it). It leaves on_stack_entry_data and js_node_test_mark_result as they are here. Whichever lands second rebases mechanically.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant