Skip to content

shell: finish a command when the wait for its process fails - #43469

Open
robobun wants to merge 6 commits into
mainfrom
robobun/ce213aa2/shell-wait-error-completes
Open

robobun wants to merge 6 commits into
mainfrom
robobun/ce213aa2/shell-wait-error-completes

Conversation

@robobun

@robobun robobun commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • await Bun.$cmd`` never settles when the wait for the command's process fails. bun stays alive and prints no error.
  • The cause is ShellSubprocess::on_process_exit (src/runtime/shell/subproc.rs:944). Its Status::Err arm is an empty // TODO: handle error block, so the function returns before Cmd::on_exit and the Cmd never finishes.

Fix

  • on_process_exit stores the wait error on the Cmd and calls Cmd::on_exit(1). The Cmd still waits for its piped stdio to close, so the output is kept. Then it writes bun: failed to wait for <argv0>: <error> to its stderr and exits with 1.
  • Correct because a failed spawn already reports this way, and Status::Err is final: Process::on_exit removes the exit handler. The exit status is lost, so the PR does not guess one.
  • Verified: test/js/bun/shell/shell-hang.test.ts (new POSIX case, release 1.4.3-canary.1 times out). Also bunshell, exec and throw.
  • Self-reviewed: the review said keep the PR, but it was cut off before I could read its full concern list. Not done on purpose: the Signaled arm of the same function, which spawn: name exit signals and send named signals with the platform's numbers #39970 fixes.

Background

  • Status (src/spawn/process.rs) is the result of a wait: Exited, Signaled, or Err. It is Err when wait4 fails (ECHILD here).
  • ECHILD means bun can no longer wait for the process: other code called waitpid(-1) first, or SIGCHLD is SIG_IGN and the kernel reaped the child.
  • A Cmd (src/runtime/shell/states/Cmd.rs) is the shell state for one command. It finishes when it has an exit code and every piped stdio is closed.
Notes

Repro on release 1.4.3-canary.1. repro.js is const r = await Bun.$/bin/echo hi.nothrow().quiet(); console.log(r.exitCode, r.stderr.toString()).

$ timeout -s KILL 8 bash -c 'trap "" CHLD; exec "$0" "$@"' bun repro.js; echo rc=$?
rc=137

With this change:

1 bun: failed to wait for /bin/echo: No child processes
rc=0

The same start with bun run --shell=bun <script> hangs on the release build. With this change it prints the message, and the script exits with 1.

What this PR does not do. Started with SIGCHLD ignored, bash, dash and Node all report the real exit status (I ran sh -c 'exit 3' under each: 3 every time). Each replaces the inherited disposition. bun keeps it, so the kernel discards the status. This PR turns the hang into a reported failure. #38022 resets an inherited SIG_IGN before the first spawn, which keeps the real status in that setup. The two changes do not depend on each other. After #38022, Status::Err still arrives when other code reaps the child, when the poll for the process cannot be registered again (Process::on_wait_pid), and on Windows when libuv reports a negative exit status (on_exit_uv). #37293 makes the second of these rarer on Linux. It does not change what the shell does when the status still arrives.

Self-review. A review of the first version of this diff ran, and the run was cut off several times. I read its verdict (keep the PR) and its arguments for and against, but not its final list of concerns, so there may be points I have not seen. From what I read, I changed these: the body claimed that every Status now reaches Cmd::on_exit, which is false while the unnamed-signal arm remains. The first test ignored SIGCHLD, which pinned a result that bash, dash and Node do not give and depended on #38022 resetting the disposition only once. It was also Linux only with no reason given. I kept the message wording: it prints the resolved path and the errno text, like the other bun: lines in Cmd.rs.

Sibling site left alone on purpose. In the same function, a Signaled status whose signal has no name (for example kill -40 $$) also returns before Cmd::on_exit. #39970 fixes that arm and adds its own case to the same test file. This PR does not touch it, so the second of the two PRs to land needs a small rebase (adjacent lines in subproc.rs, the import line and the end of shell-hang.test.ts).

Why exit 1 and a message, not a rejected promise. The closest case in the shell is a spawn that fails: Cmd::transition_to_exec reports it through Builtin::cmd_write_failing_error, which writes to the stderr of the command and finishes it with 1. The wait error uses the same function. ||, ;, pipelines and .nothrow() then behave as they do for every other failed command, and the captured stdout is kept. Without .nothrow() the promise rejects with a ShellError (exit code 1, the message in stderr).

Why the message is written in Done. cmd_write_failing_error finishes the Cmd when the write completes. It does not wait for the subprocess pipes. Cmd::finish_if_done sets Done only after the exit code is in and every piped stdio is closed, so Done is the first point where it is safe to finish through that function. The message therefore comes after the stderr of the command itself.

The test. A child bun starts sh -c "echo out; echo err >&2; exit 3" with Bun.$, then calls a blocking waitpid(-1) through bun:ffi before it returns to the event loop. That call always reaps sh first (it reads exit status 3), so the wait in bun fails with ECHILD. The child does this with .quiet() (stderr is captured only) and without it (stderr also goes through the IOWriter to the real stderr, which is the WaitingWriteErr state). Both must give exit code 1, out\n on stdout, and err\n plus the message on stderr. The test does not depend on the SIGCHLD disposition, so #38022 does not change it. It does depend on bun waiting on the JS thread. Without pidfd_open (a seccomp profile can block it), bun on Linux sets its waiter-thread flag itself (src/spawn_sys/spawn_process.rs:544) and waits on a second thread, which reaps sh first. With that mode forced, the first version of this test failed in 30 of 30 runs (reaped: false, exit code 3). So on Linux the test process calls pidfd_open once through bun:ffi, and the case uses test.skipIf when it fails. The runner then reports a skip, not a pass. The test also removes BUN_FEATURE_FLAG_FORCE_WAITER_THREAD from the child env. I checked this on a host without pidfd_open: a seccomp filter that returns EPERM for syscall 434, installed before exec (x86_64 only, EPERM only). With the filter the file reports 6 pass, 1 skip, 0 fail. With the filter and no force flag, the first version of the test body gives reaped: false and exit code 3 in 10 of 10 runs, so bun does fall back by itself. Without the filter the case runs and passes. The case is serial on purpose, unlike the other cases in the file. On a regression its child bun never exits. On the release build, the runner kills that child when the timed-out test is serial (killed 1 dangling process, no child left) and leaves it alive when the test is concurrent. On the release build the command never settles and the test times out. I ran it on Linux only. In CI build 118353 every Linux test lane passed it (glibc, musl, x64, aarch64, ASAN). On macOS both lanes passed in CI build 118400, which is the head (bf4ee422cd): aarch64 and x64, 2 of 2 test shards each. The case is not skipped on macOS, so it ran and passed there. macOS aarch64 also passed in build 118385. macOS x64 finished only in build 118400, so that is one run. macOS never uses the waiter thread.

Builtin.rs. The only change there removes a sentence from the cmd_write_failing_error doc that listed its callers. The list was already incomplete before this PR (it left out Cmd::child_done).

The 700 ms timeouts. The six existing cases in shell-hang.test.ts each spawn bun and had a 700 ms timeout. On a loaded machine the debug build needs 0.5 to 1.9 s per case, so the file failed with no hang. The second commit removes the explicit timeouts. The default timeout still catches a hang.

Suites run on the debug (ASAN) build. shell-hang, bunshell, exec, throw, lazy, yield, shell-pipe-read-fault, shell-write-fault. One of five bunshell.test.ts runs had a single failure that did not repeat. I did not capture which test it was.

Also run on the debug (ASAN) build. A script with the wait error in a pipeline, ||, &&, ;, a subshell, an if condition, < ${buffer}, > ${buffer}, > file, .text(), and a command that throws ShellError. Every case completes with the command output kept. Twenty iterations with detect_leaks=1 report no leak. Forty failing waits leave the open fd count unchanged (9 before, 9 after). cargo clippy -p bun_runtime reports nothing for the changed files.


no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/shell/shell-hang.test.ts

ShellSubprocess::on_process_exit ignored Status::Err, so Cmd::on_exit
never ran and the Bun.$ promise never settled. The command now exits
with 1 and reports the wait error on its stderr, like a failed spawn.
Each case spawns bun. On a loaded machine the debug build needs more
than the 700 ms these cases allowed, so the file failed without a hang.
@coderabbitai

coderabbitai Bot commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: oven-sh/bun/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: cdf8862d-60ab-488e-927e-8e70d6721001

📥 Commits

Reviewing files that changed from the base of the PR and between 26e7a4b and 9209b49.

📒 Files selected for processing (4)
  • src/runtime/shell/Builtin.rs
  • src/runtime/shell/states/Cmd.rs
  • src/runtime/shell/subproc.rs
  • test/js/bun/shell/shell-hang.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 0 remain after this review.


Walkthrough

The shell now stores subprocess wait errors, reports them during command completion, and returns exit status 1. Shell tests cover quiet and loud execution, propagated ECHILD errors, reaped processes, and platform-specific skips.

Changes

Shell wait-error handling

Layer / File(s) Summary
Capture and report wait errors
src/runtime/shell/states/Cmd.rs, src/runtime/shell/subproc.rs, src/runtime/shell/Builtin.rs
Status::Err stores the subprocess error on Cmd. CmdState::Done reports the stored error and uses exit status 1. Caller documentation now lists the updated paths.
Regression test coverage
test/js/bun/shell/shell-hang.test.ts
Tests cover forced wait failures, quiet and loud output, exit status, reaped status, and ECHILD. Linux skips when pidfd_open is unavailable.

Suggested reviewers: dylan-conway

Priority: ⬇️ Low

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed The description clearly explains the problem, implementation, verification steps, test coverage, and known scope. It does not use the exact template headings, but it provides the required information …
Title check ✅ Passed The title clearly and concisely describes the main change: completing a command when waiting for its process fails.

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: the fix and a test are in this PR.

How I reproduced it, on release 1.4.3-canary.1 (Linux x64):

$ cat repro.js
const r = await Bun.$`/bin/echo hi`.nothrow().quiet();
console.log(r.exitCode, r.stderr.toString());
$ timeout -s KILL 8 bash -c 'trap "" CHLD; exec "$0" "$@"' bun repro.js; echo rc=$?
rc=137

The command never settles, so timeout kills bun. With this branch the same run prints 1 bun: failed to wait for /bin/echo: No child processes and exits 0.

The new case in test/js/bun/shell/shell-hang.test.ts produces the same wait failure another way: the child bun reaps the command first with a blocking waitpid(-1) through bun:ffi. It times out on the release build and passes on the debug build with the fix.

CI on bf4ee42 (build 118400) is final. test/js/bun/shell/shell-hang.test.ts passed on every lane where it runs: all Linux lanes (glibc, musl, x64, aarch64, ASAN) and both macOS lanes (aarch64 and x64, 2 of 2 test shards each). The new case is not skipped on macOS, so it ran there. macOS x64 finished for the first time in this build, so that is one run.

The build is red because of one file outside this diff: test/js/bun/spawn/spawn.test.ts on debian 13 x64-asan, case an idle reader stopped at the highwater mark does not keep the process alive (trickling writer). It asserts that a child bun has an empty stderr, and the child printed LeakSanitizer's exit warning (ptrace appears to be blocked, Child exited with signal 42). That child is not part of the shell test's process tree, and this PR does not touch Bun.spawn. The failure comes and goes: it was red in 2 of the 4 builds of this PR, and the CI helper showed it as also failing on main an hour ago and not now. An earlier version of this comment cited that tag, which has since changed. The other failures in the build passed on retry. I have not used a CI retrigger.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline findings, I also traced the new wait_error field from on_process_exit to its consumer: finish_if_done is the only producer of CmdState::Done for subprocess-backed Cmds and always yields Next(this_id), so the Done arm in Cmd::next runs on both the exit-first and stdio-closes-last orderings, and the take() plus WaitingWriteErr -> child_done(.., 1) path means the error is reported exactly once. The Cmd::new constructor is the only initializer site and was updated.

Extended reasoning...

Findings were reported inline (the waiter-thread test race and the sibling unnamed-signal arm in on_process_exit), so this body only records what else was examined. I read src/runtime/shell/states/Cmd.rs around on_exit/finish_if_done (lines 1022-1048) and the Done arm of next (lines 299-320): finish_if_done sets Done only when the exit code is present and every piped stdio has closed, then yields into the state machine, so the new wait_error check is reached regardless of whether the wait failure or the last pipe close arrives first. cmd_write_failing_error transitions to WaitingWriteErr and its completion calls child_done(parent, this, 1) (line 344), matching the existing spawn-error path. The only constructor site (Cmd::new, line 222) initializes the field. Nothing here is a guarantee of correctness; the inline findings still warrant a human look.

Comment thread test/js/bun/shell/shell-hang.test.ts
Comment thread src/runtime/shell/subproc.rs
Without pidfd_open, bun on Linux waits for children on a second thread.
That thread reaps the command before the waitpid(-1) in the test, so the
wait never fails. The child probes pidfd_open and skips when it is not
available, and the test drops BUN_FEATURE_FLAG_FORCE_WAITER_THREAD.
Comment thread src/runtime/shell/Builtin.rs Outdated
Comment thread src/runtime/shell/states/Cmd.rs
The list was already incomplete (it left out Cmd::child_done) and goes
stale with each new call site.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline finding, I also checked the new Done branch in src/runtime/shell/states/Cmd.rs for a re-entry loop or double finish: wait_error is take()n before cmd_write_failing_error, the Fd path's completion (on_io_writer_chunk) goes straight to child_done(1) without re-entering Done, and the Pipe (.quiet()) path appends to the captured stderr and finishes synchronously — so each failed wait reports exactly once.

Extended reasoning...

The Status::Err arm in subproc.rs now yields exit code 1 and stores the error on the Cmd using the same handle.cmd_mut() write the existing interrupted flag already uses, before on_exit, so no new pointer-lifetime shape is introduced. In Cmd.rs, the Done arm consumes wait_error once via take(); Builtin::cmd_write_failing_error either enqueues to the IOWriter and moves to WaitingWriteErr (whose completion callback calls child_done(parent, this, 1) directly) or, for captured stderr, appends and calls child_done synchronously. Neither path re-enters the Done match, so there is no loop or duplicate report. The argv0 slice uses saturating_sub(1) to strip the NUL and falls back to <unknown> for empty args. The remaining concern is the test's silent early return on hosts without pidfd_open, which is already posted inline.

Comment thread test/js/bun/shell/shell-hang.test.ts Outdated
@robobun

robobun commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 12:20 PM PT - Sep 19th, 2026

❌ @robobun, your commit bf4ee42 has 1 failures in Build #118400 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 43469

That installs a local version of the PR into your bun-43469 executable, so you can run:

bun-43469 --bun

The case returned early when the child found no pidfd_open, which the
runner counts as a pass. The test process now probes pidfd_open once and
gates the case with test.skipIf, so the runner reports a skip.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Comment thread test/js/bun/shell/shell-hang.test.ts
When the case times out its child bun never exits. The runner kills the
child of a timed-out serial test and leaves the child of a concurrent one.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant