Skip to content

test: keep --watch alive when --bail stops a run - #35094

Closed
robobun wants to merge 4 commits into
mainfrom
farm/bf5e6d49/test-watch-bail-exits-watcher
Closed

robobun wants to merge 4 commits into
mainfrom
farm/bf5e6d49/test-watch-bail-exits-watcher

Conversation

@robobun

@robobun robobun commented Jul 22, 2026 •

Copy link
Copy Markdown
Collaborator

What

bun test --watch --bail exits the whole watch process on the first bail-out. The edit that would fix the failing test never re-runs. Without --bail the same watcher survives and reruns on the next edit.

Repro

mkdir d && cd d
cat > b.test.ts <<'EOF'
import { test, expect } from "bun:test";
import { v } from "./v";
test("t1", () => { expect(v).toBe(1); });
test("t2", () => { expect(2).toBe(2); });
EOF
echo 'export const v = 0;' > v.ts
echo '{}' > package.json
bun test --watch --bail
# => prints 'Bailed out after 1 failure' and exits 1 instead of watching

Cause

CommandLineReporter::handle_test_completed calls Global::exit(1) directly when the fail count reaches the bail threshold, inside the per-test completion callback. Under --watch that kills the process before control reaches run_event_loop_for_watch. The module-load-rejection bail path in TestCommand::run has the same hard exit.

The parallel coordinator already handles this correctly: it sets a bailed flag and drains, returning normally.

Fix

Give the serial runner the same flag-based bail:

  • Execution.aborted stops the step loops (step_group, step_group_one, step's per-group loop) so no further tests in the current file run.
  • CommandLineReporter.bailed stops the per-file loop in run_all_tests and the --rerun-each repeat loop in TestCommand::run.

Under --watch the bail path now prints the banner, sets the flags, and lets the run unwind back to exec, which prints the full summary and enters run_event_loop_for_watch. When the watched file changes, reload_process re-execs as before. When not watching, the existing Global::exit(1) is unchanged.

Verification

# fail-before (released bun): process exits, never sees the fixing edit
USE_SYSTEM_BUN=1 bun test test/cli/test/bun-test.test.ts -t 'keeps watching after a bail'
# pass-after
bun bd test test/cli/test/bun-test.test.ts -t bail

The test spawns bun test --watch --bail with a failing test, waits for the bail banner and the end-of-run summary on stderr, then fixes the dep file and waits for 2 pass from the re-run. Skipped on Windows where --watch uses a respawning parent instead of exec-in-place (same as the existing --changed --watch test).


no test proof · iteration 2 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/cli/test/bun-test.test.ts

Supersedes #19918 (same approach against the pre-Rust Zig sources, which no longer exist).

Fixes #20318
Fixes #6453

With --watch --bail, the first failing run printed the bail banner and
called Global::exit(1) from inside the per-test completion callback,
terminating the watcher. The fixing edit never re-ran.

Give the serial runner the same flag-based bail the parallel
coordinator already has: Execution.aborted stops the per-file step
loops and CommandLineReporter.bailed stops the per-file and
--rerun-each loops, so control returns to TestCommand::exec. Under
--watch that reaches run_event_loop_for_watch; otherwise the existing
Global::exit(1) path is unchanged.
@coderabbitai

coderabbitai Bot commented Jul 22, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Changes

Watch-mode bail handling

Layer / File(s) Summary
Execution abort propagation
src/runtime/test_runner/Execution.rs
Adds an aborted state and stops top-level, group, and concurrent test sequencing when set.
Watch-aware reporter bail handling
src/runtime/cli/test_command.rs
Tracks bail state, aborts the current execution, exits in non-watch mode, and unwinds for watch-mode reruns.
Watch and bail integration test
test/cli/test/bun-test.test.ts
Validates that --watch --bail stops the failed run and reruns successfully after a file fix.

Possibly related PRs

  • oven-sh/bun#34443: Both changes modify test-runner early-exit and cleanup behavior.

Suggested reviewers: jarred-sumner

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes implement watch-aware bail handling and a regression test, matching the linked issues' requirements.
Out of Scope Changes check ✅ Passed The patch stays focused on watch-and-bail behavior and the related regression test, with no obvious unrelated changes.
Title check ✅ Passed The title clearly summarizes the main change: keeping watch mode alive when bail stops a test run.
Description check ✅ Passed The description covers what changed and how it was verified, though it uses custom headings instead of the template's exact section names.

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Jul 22, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 5:18 AM PT - Jul 22nd, 2026

❌ @robobun, your commit e16c003 has 2 failures in Build #77648 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 35094

That installs a local version of the PR into your bun-35094 executable, so you can run:

bun-35094 --bun

@github-actions

Copy link
Copy Markdown
Contributor

Found 2 issues this PR may fix:

  1. bun test: --watch and --bail should work together #20318 - Requests --watch and --bail to work together; this PR prevents bail from killing the watch process
  2. Allow bun test to support both the --watch and --bail flags at the same time. #6453 - Duplicate of bun test: --watch and --bail should work together #20318, same request for --watch and --bail compatibility

If this is helpful, copy the block below into the PR description to auto-close these issues on merge.

Fixes #20318
Fixes #6453

🤖 Generated with Claude Code

@github-actions

Copy link
Copy Markdown
Contributor

This PR may be a duplicate of:

  1. fix(test): bail correctly halts current run in watch mode #19918 - Same fix for --bail exiting the process instead of keeping the watcher alive in bun test --watch --bail, but targets the old Zig sources (pre-Rust rewrite)

🤖 Generated with Claude Code

Comment thread src/runtime/cli/test_command.rs
Comment thread src/runtime/cli/test_command.rs Outdated
Comment thread test/cli/test/bun-test.test.ts

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three prior nits are now addressed (the fixture tests were renamed to "test #1"/"test #2", and the two bail sites now share the hoisted let watching = ... shape with summary → banner ordering preserved for non-watch). No new issues found. Deferring to a human for the control-flow change itself — flag-based unwind through Execution.aborted and CommandLineReporter.bailed across three step loops, the per-file loop, and the --rerun-each loop.

What was reviewed:

  • Non-watch --bail path is byte-for-byte behavior-preserving at both sites (handle_test_completed and the module-load-rejection arm in TestCommand::run).
  • aborted guards in step_group, step_group_one, and step's per-group loop return Complete before starting the next sequence — same effective skip set as the old Global::exit(1).
  • New test polls stderr for observable conditions (no sleeps), asserts test #2 never ran, then edits the dep and awaits 2 pass from the re-run; await using proc covers cleanup.
  • The two unrelated CI failures (test-gc-http-client-connaborted.js, test-net-connect-memleak.js) are pre-existing flakes on x64 Linux, not touched by this diff.
Extended reasoning...

Overview

The PR changes bun test --watch --bail so that hitting the bail threshold no longer calls Global::exit(1) and kills the watcher. Instead it sets two flags — CommandLineReporter.bailed (stops the per-file loop in run_all_tests and the --rerun-each repeat loop) and Execution.aborted (stops step_group, step_group_one, and the per-group loop in Execution::step) — so the run unwinds back to exec, which prints the summary and enters run_event_loop_for_watch. The non-watch path is unchanged: it still prints summary → banner → junit → Global::exit(1). The module-load-rejection bail arm in TestCommand::run gets the same treatment. A new test in test/cli/test/bun-test.test.ts spawns bun test --watch --bail, waits for the bail banner and the end-of-run summary, asserts the second test never ran, edits the failing dep, and waits for 2 pass from the re-run.

Security risks

None. This is test-runner control flow; no untrusted input, no auth/crypto/permissions surface.

Level of scrutiny

Moderate. The change is small (~60 lines of Rust across two files, plus a test) and follows the same flag-and-drain pattern the parallel coordinator already uses, but it threads a new early-exit through five loop sites in the serial runner. Getting the unwind order wrong could leave the watcher in an odd state or change what runs after bail. I traced each guard: aborted is set inside handle_test_completed (called from on_sequence_completed inside advance_sequence), and every loop that could start a next sequence/group checks it at the top before doing so, so no additional test bodies run — matching the old hard-exit's effective behavior. Under --watch the next file-change triggers reload_process (exec-in-place), so any skipped afterAll groups on the bailed run don't leak into the next run. That all looks correct, but it's the kind of multi-site control-flow change a maintainer should eyeball.

Other factors

I left three nits on earlier revisions (non-watch output-order regression at the module-load bail site, duplicated banner block, and a ~0.13% flake from not.toContain("t2") colliding with the mkdtemp suffix). All three are fixed in the current diff — the two bail sites now bind watching once and print summary before the banner when not watching, and the fixture tests are named "test #1"/"test #2" with not.toContain("test #2"), matching the sibling --bail tests. The 30s per-test timeout was flagged and ruled out (it's a spawned --watch subprocess awaiting a filesystem-triggered rerun; a deadline is appropriate and await using proc guarantees kill on failure). The two CI failures are unrelated Node-parallel GC/memleak flakes on Linux x64. Given this is a real control-flow change to the test runner rather than a mechanical edit, I'm deferring rather than approving.

@robobun

robobun commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator Author

Diff is green. The only red CI lanes across the last three builds (77612, 77626, 77648) are two unrelated Node.js GC-timing tests on Linux x64 that don't go through the test runner at all:

  • test/js/node/test/parallel/test-net-connect-memleak.js
  • test/js/node/test/parallel/test-gc-http-client-connaborted.js

Both are bun <file> style (not bun test), so test_command.rs / Execution.rs aren't in their path. They fail the same way on all three builds regardless of the commit here.

The new --bail with --watch test passes on every completed lane. All review threads addressed. Ready for a maintainer.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond the inline nit, I traced a few things and they check out: the non-watch --bail path is behavior-identical (same summary→banner→Global::exit(1) order after f2d9e07); Execution.aborted's StepResult::Complete reaches Phase::Done via the same _advance path as natural completion; and neither bailed nor aborted leak across watch reruns (watch reloads via re-exec, and each file's Execution::init() starts aborted: false regardless).

Extended reasoning...

Verified the three earlier review comments were addressed in f2d9e07 and e16c003. Traced the new aborted checks in step_group/step_group_one/step — they're read-only early-returns at loop heads that funnel into the existing StepResult::Complete → _advance completion path, and the fresh .get() reborrow in step_group_one matches the file's existing stacked-borrows pattern. The run_all_tests per-file check and the --rerun-each break correctly unwind to exec, which prints the summary at line 2878 and enters run_event_loop_for_watch. The two CI failures (test-gc-http-client-connaborted.js, test-net-connect-memleak.js) are unrelated to this change.

Comment thread test/cli/test/bun-test.test.ts
@robobun

robobun commented Sep 13, 2026

Copy link
Copy Markdown
Collaborator Author

Closing as part of a cleanup of stale pull requests. This PR has had no new commits since 2026-07-22, it conflicts with main, and its last CI run failed. This is not a judgment on the fix itself. The linked issue (#6453) stays open. If the problem still reproduces on a current build, reopen this PR after a rebase or open a new one against main.

@robobun robobun closed this Sep 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow bun test to support both the --watch and --bail flags at the same time. bun test: --watch and --bail should work together

2 participants