Repository navigation
bun test --parallel: act on SIGINT/SIGTERM immediately instead of at the next housekeeping timer - #39039
bun test --parallel: act on SIGINT/SIGTERM immediately instead of at the next housekeeping timer#39039robobun wants to merge 4 commits into
Conversation
…ERM handler The abort handler only set SHOULD_ABORT. Coordinator::drive reads the flag between polls, and the poll retries on EINTR, so once every worker was mid-file the flag was noticed only when a housekeeping timer (GC repeating timer, Date header timer, both ~1s) happened to wake the loop: Ctrl+C or SIGTERM took up to a second to act on. Publish the coordinator's uws loop when the handler is installed and have the handler (POSIX signal handler and Windows console control handler) call us_wakeup_loop after setting the flag, so the poll returns immediately. ensure_waker() moves ahead of the handler install since it is what sets the loop pointer up.
|
Updated 12:05 PM PT - Aug 15th, 2026
❌ @robobun, your commit 7988c6b has some failures in 🧪 To try this PR locally: bunx bun-pr 39039That installs a local version of the PR into your bun-39039 --bun |
|
Warning Review limit reached
Next review available in: 25 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (3)
Comment |
|
Status: reproduced and fixed, waiting on CI and a maintainer.
|
There was a problem hiding this comment.
Beyond the inline nit, I checked async-signal-safety of us_wakeup_loop — on the epoll path it's an atomic add plus write(eventfd), and the same primitive is already called from signal context by PosixSignalHandle::enqueue → EventLoop::wakeup, so this follows established precedent. Also checked: no lost-wakeup window (flag is stored before the wakeup, eventfd/mach-port stays signalled if the poll hasn't been entered yet), and the ensure_waker() reordering is required on Windows since usockets_loop() panics before it runs.
Extended reasoning...
Verified the signal-context call path against the existing PosixSignalHandle::enqueue pattern (src/jsc/PosixSignalHandle.rs:67 → EventLoop::wakeup → us_wakeup_loop), confirmed us_internal_async_wakeup on Linux/macOS/FreeBSD is a raw syscall with no allocation or locks, and confirmed usockets_loop() requires ensure_waker() to have run first on Windows (src/jsc/event_loop.rs:642). The uninstall path clears LOOP after restoring the previous sigaction, so a late signal either reaches the restored handler or finds a still-valid loop pointer. Nothing beyond the timeout nit surfaced.
There was a problem hiding this comment.
I reviewed this PR and didn't find any bugs. The change is small and follows the existing AtomicPtr + raw us_wakeup_loop pattern used elsewhere (e.g. js_bundle_completion_task.rs, MiniEventLoop::wakeup), and the earlier timeout feedback has been addressed. Because it modifies a signal handler with async-signal-safety reasoning across three platform backends (eventfd write / mach_msg / kevent64 / uv_async_send), and the Windows path was only hand-verified, a human look would still be worthwhile.
What was reviewed:
LOOPstore/clear ordering vs.sigactioninstall/restore — pointer is published before the handler is installed and cleared after it is removed, so no window where the handler runs with a stale/null pointer that matters.us_wakeup_loopimplementation on each backend — atomic increment +write()to eventfd (Linux),mach_msg(macOS),kevent64NOTE_TRIGGER (FreeBSD),uv_async_send(Windows, from the console-control thread, not signal context); no allocation or locks.ensure_waker()reorder inrunner.rs— needed sousockets_loop()returns non-null beforeinstall(); the old call site is removed so it still runs exactly once.- New test — fail-before condition made deterministic on Linux via
/proc/<pid>/statuscontext-switch polling; 15000 ms timeout now matches the sibling signal test.
Extended reasoning...
Overview
This PR makes bun test --parallel's coordinator react to SIGINT/SIGTERM immediately instead of waiting up to ~1s for the next housekeeping timer. Three files touched: Coordinator.rs (the abort_handler module now stores the uws loop pointer in an AtomicPtr and calls the raw us_wakeup_loop extern from the signal handler / Windows console-control handler after setting the flag), runner.rs (moves ensure_waker() a few lines earlier and passes the loop pointer to install()), and parallel.test.ts (a new .concurrent.each test over SIGTERM/SIGINT that times how long the coordinator takes to print its "Interrupted" line).
Security risks
None. This is test-runner shutdown handling; no untrusted input, no auth/crypto/permissions surface. The only correctness-sensitive aspect is async-signal-safety.
Level of scrutiny
Medium-high. The diff is small (~30 lines of Rust plus a test), but it changes what runs in POSIX signal context. I verified: (1) us_wakeup_loop on Linux is an atomic add plus write() to an eventfd — write() is async-signal-safe, and the eventfd write essentially always succeeds so the errno-checking retry loop is not reached in practice; on macOS it is mach_msg; on FreeBSD it is a kevent64 NOTE_TRIGGER submit; on Windows the handler runs on a dedicated console-control thread so signal-safety does not apply and uv_async_send is the documented thread-safe primitive. (2) The AtomicPtr<Loop> + raw-extern pattern is exactly what js_bundle_completion_task.rs, HTTPThread::wakeup, and MiniEventLoop::wakeup already use, and bun_uws::us_wakeup_loop is publicly re-exported for exactly this reason (to avoid forming &mut Loop off the owning thread). (3) The LOOP.store in install() happens before sigaction(), and uninstall() restores sigaction() before clearing LOOP, so the handler never observes a torn state; the null check makes a race benign anyway. (4) The ensure_waker() reorder is required because usockets_loop() panics on null; the old call site is removed so it still runs once.
One minor point I considered but did not flag: the Linux eventfd path in us_internal_async_wakeup reads errno on the (essentially unreachable) EINTR/EAGAIN branch, and the handler does not save/restore errno. Since this is a terminal shutdown path (the process exits with 130 on the very next tick) and the eventfd write of a single 8-byte value never fails in practice, this is not a real concern here.
Other factors
- The earlier inline comment about the missing per-test timeout was addressed in 4a6fc0f (15000 ms, matching the sibling test).
- The comment-cop bot's complaints about the module doc were addressed in 7988c6b (trimmed to four lines, shorter than what it replaces).
- The test uses the file's established conventions (
tempDir,bunEnvspread, PID file polling like the sibling test,test.concurrent.each,skipIf(isWindows)). It measures time-to-first-stderr-byte rather than time-to-exit so ASAN teardown stays out of the number, and thewaitForCoordinatorToParkhelper makes fail-before deterministic on Linux by aligning the signal to the start of a quiet period. - The Windows path is exercised only by hand (CTRL_BREAK_EVENT via a Python driver, per the PR body); CI cannot synthesize console control events. Given that the Windows change is one line (
SHOULD_ABORT.store→request_abort()) anduv_async_sendfrom a non-main thread is the standard libuv wakeup, this seems low-risk, but a maintainer familiar with the Windows CI should confirm.
Deferring rather than approving because signal-handler changes across three eventing backends are the kind of thing a maintainer who knows usockets internals should sign off on, even when the diff is small and the reasoning checks out.
|
On the errno question from the review: the handler leaves errno alone on purpose. None of the wakeup backends touch it on the path that actually runs (the eventfd write cannot block or be interrupted, so its retry branch is unreachable; mach_msg does not use errno; kevent only sets it on failure), and the existing signal handlers in the tree (Bun__onPosixSignal, the SIGCHLD waiter wakeup) follow the same convention, so adding per-platform errno save/restore here would be code for a path that cannot be reached. |
…awned (#41467) ### Problem - `bun test --parallel` exits 130 for any abort signal. `abort_all` in `src/runtime/cli/test/parallel/Coordinator.rs:234` has `self.aborted = Some(130)`. SIGTERM should give 143, which is what a serial `bun test` (killed by the default disposition) and a shell report. - A worker that dies mid-file (`process.exit`, SIGKILL, a crash) leaves the processes its test spawned running. The file is counted as `worker crashed`, the run goes on, and the `sleep 777` from the test is still there after `bun test` exits. Only the abort path (`kill(-pid, SIGTERM)`) cleaned up descendants. ### Fix - The signal handler stores the signal number in `ABORT_SIGNAL` (an `AtomicI32`, 0 when none) instead of a flag. `abort_all` exits with `SignalCode::to_exit_code()`, the `128 + signal` helper `filter_run` and `multi_run` already use. The Windows console handler stores SIGINT, so the result there is 130 as before. - `reap_worker` sends SIGTERM to the worker's process group when the worker had a file in flight, through the same `terminate_process_group` helper the abort paths now share. The worker is the group leader, so the group reaches everything its tests spawned. Unix only: on Windows the coordinator's kill-on-close Job Object already covers descendants. - Correct because the pid of a dead group leader stays reserved while any member of the group lives, so `kill(-pid)` after the exit cannot hit an unrelated process. If the group is empty the call fails with ESRCH and nothing happens. - Verified: `test/cli/test/parallel.test.ts` (two new cases, 1.4.3 fails both). Also `parallel-startup-failure.test.ts` and the JUnit suites. ### Background - Under `--parallel` a coordinator spawns N worker processes, each in its own process group (`new_process_group: true` in `Worker.rs`), and dispatches test files to them over an IPC channel. - Workers run with `--isolate`. Between files the isolate swap kills the subprocesses the previous file spawned. A worker that exits during a file never gets there. - `reap_worker` runs when the coordinator has drained a worker's IPC pipe after its exit. It accounts the in-flight file as crashed and, if files remain, spawns a fresh worker. <details><summary>Notes</summary> Probes on bun 1.4.3 (`kill -TERM` on the coordinator 1 s into a run): ``` serial: rc=143 parallel: rc=130 SIGINT parallel: rc=130 ``` Descendants after a mid-file death, `--parallel=2`, three files: a spawns `sleep 777` then `process.kill(process.pid, "SIGKILL")`, b spawns `sleep 778` then `process.exit(0)`, c spawns `sleep 779` and passes: ``` 1 pass 2 fail PID PGID PPID ARGS 402 399 1 sleep 777 409 404 1 sleep 778 ``` `sleep 779` was gone: the isolate swap after c killed it. 777 and 778 outlive the run. `--no-orphans` (`BUN_FEATURE_FLAG_NO_ORPHANS=1`) is an opt-in that covers the same case with a kernel SIGKILL (PDEATHSIG) on Linux. It is process-wide and also kills `detached` children, so the runner scopes it away from tests that expect a detached grandchild to survive. This change is the default path: the SIGTERM that `--isolate` already sends between files, extended to a worker that dies mid-file. The new test clears `BUN_FEATURE_FLAG_NO_ORPHANS` because the ASAN lanes set it, and PDEATHSIG would kill the grandchildren before the path under test runs. #39039 (open) rewrites the same `abort_handler` store sites and asserts 130 for SIGTERM. Whichever lands second needs a small rebase, and that assertion should become 143. Self-reviewed: 4 concerns raised, 3 addressed (`SignalCode::to_exit_code`, one shared kill helper, the `--no-orphans` note above). The #39039 ordering is recorded here for the reviewer. The grandchild in the new test records its own termination through a SIGTERM handler. After the worker dies the grandchild is reparented, and an init that does not reap would leave a zombie that still answers `kill(pid, 0)`. The existing test `--parallel: each worker has a unique JEST_WORKER_ID` fails on a debug build with or without this change. Its 200 ms fixture sleep is shorter than worker startup under a debug build, so one worker takes two files. </details>
Problem
bun test --parallel(2+ files, so the process-pool coordinator is in use) takes up to a second to act on Ctrl+C / SIGTERM once every worker is in the middle of a file: measured 950-970 ms fromkillto exit, 5/5 runs, with the current release build.abort_handler::install(src/runtime/cli/test/parallel/Coordinator.rs) only storesSHOULD_ABORT.Coordinator::drivereads the flag between polls, and while it is parked inauto_tick()nothing reads it.bun_epoll_pwait2/bun_kevent64_wait(packages/bun-usockets/src/eventing/epoll_kqueue.c) retry on EINTR for the remaining timeout.Fix
abort_handler::installnow takes the coordinator's uws loop and publishes it in anAtomicPtr; the POSIX signal handler and the Windows control handler set the flag and then callus_wakeup_loopon it, so the polldrive()is parked in returns at once and the existing flag check runs.uninstallclears the pointer.us_wakeup_loopis an atomic increment plus aneventfdwrite (Linux) /mach_msg(macOS) /keventNOTE_TRIGGER (FreeBSD) /uv_async_send(Windows): no allocation, no locks, so it is fine in signal context. It is the same primitiveprocess.on(<signal>)delivery (PosixSignalHandle::enqueue) and every other cross-thread wakeup of this loop already use, called through the raw extern asHTTPThread::wakeupandMiniEventLoop::wakeupdo, so no&mut Loopis formed while the main thread is inside the tick.drive()'s flag check and the poll still wakes the poll, because the eventfd / mach port / async is already signalled when the poll is entered (andpending_wakeupsmakes the kqueue path poll non-blocking).ensure_waker()moves ahead of the handler install in runner.rs, since it is what sets up the loop pointer the handler needs; it used to run a few lines later, beforedrive().test/cli/test/parallel.test.ts, "coordinator acts on SIGTERM/SIGINT immediately while every worker is mid-file". Without the src change it fails with ~901 ms (release) / ~935 ms (debug ASAN); with it the coordinator reports the interruption within a few ms (about 1 ms measured, ~35-65 ms to full exit under ASAN). The existing "SIGTERM on coordinator kills workers and their grandchildren" test still passes.SIGBREAKlistener. Current canary reports the interruption after 929 / 931 / 530 ms depending on when the event is sent; this branch's debug build after 1 ms in every run.voluntary_ctxt_switchesmoving, then 100 ms of quiet) so the signal is sent at the start of the quiet period. That turns the unfixed 0-1 s into a reliable ~900 ms instead of a value that passes the assertion some of the time; the fixed coordinator reacts immediately regardless of phase, so the alignment only matters for fail-before. Skipped on Windows, whereproc.killisTerminateProcessand never reaches the control handler.bun run --filter(src/runtime/cli/filter_run.rs) andbun run --parallel/--sequential(src/runtime/cli/multi_run.rs) carry their own copies of this flag-only handler and have the same defect in a worse form (their loop has no timers at all, so a SIGINT sent to the parent alone is ignored until a child exits; a terminal Ctrl+C still works there because it also hits the children). Their installers differ on purpose (SIGINT only, SA_RESETHAND so a second Ctrl+C kills immediately during the graceful abort), so this PR stays with the coordinator and the twobun runsites are tracked separately rather than folded into a shared module.Background
--paralleland 2+ files, thebun testprocess does not run tests itself; it spawns worker processes, hands them files over an IPC socketpair and streams their results. Its main loop (Coordinator::drive) is: checkSHOULD_ABORT, then block in the event loop (auto_tick, i.e.epoll_pwait2/kevent64/uv_runwith the next timer deadline as the timeout) until I/O or a timer.us_wakeup_loop: usockets' cross-thread "make the loop's poll return" primitive. Every loop owns a wakeup async (an eventfd on Linux, a mach port on macOS, an EVFILT_USER on FreeBSD, auv_async_ton Windows) registered in its poll set; signalling it makes the blocked poll return, after which the caller's own loop logic runs again.Measurements
Release 1.4.0 canary, coordinator + 2 workers parked in
Bun.sleep(60000), time fromproc.killto exit:Coordinator main-thread wakeups sampled via
/proc/<pid>/statuswhile parked (release build): one burst at +974 ms, +1975 ms, +2976 ms, nothing in between. gdb on the debug build shows the timers firing in those bursts areDateHeaderTimerandGcRepeating.Fixed debug ASAN build, time to first stderr byte ("Interrupted ...") vs exit: 0.4 / 1.0 / 0.8 ms and 40 / 35 / 65 ms. Under a 2x CPU oversubscription the exit figure went up to 350 ms, which is why the test measures the report and not the exit.
Windows (Server 2019 x64). A Python driver starts the coordinator with
CREATE_NEW_PROCESS_GROUP, waits for both workers to be inside their test (each test file doesprocess.on("SIGBREAK", () => {})so the workers, which share the group, survive the event), sendsCTRL_BREAK_EVENTto the group and times the first stderr byte and the exit: