PipeReader: do not read an fd again after a read on it fails - #43900
Conversation
PosixBufferedReader delivers the bytes read before a failed read, then the error. The delivery resolves a parked pull, and the consumer pulls again from inside it. The reader then read the fd past the error, so the error arrived after bytes read later, or never when those reads reached EOF. read_once now sets PosixFlags::READ_FAILED on a fatal error and begin_read refuses to read while it is set. The pull from inside the delivery parks, and on_reader_error rejects it. The test shim can cap each recv() and fail one with EAGAIN. Two new cases use that to put the failing recv() after bytes in the same wakeup with a pull parked.
|
Status Reproduced on debian x64, with no alpine machine.
PR: #43900 |
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository: oven-sh/bun/.coderabbit.yaml Review profile: ASSERTIVE Plan: Essentials Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan provides up to 10 included reviews per hour; 4 remain after this review. WalkthroughThe POSIX pipe reader now stops starting reads after a non-retry read error. The syscall-fault tests configure receive behavior and verify that Bun streams and ChangesPipe reader error handling
Suggested reviewers: Priority: ➖ Normal Merge Risk: ⚪ Minimal · up to No actionable merge-blocking risk is established for this change after normal checks. 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
There was a problem hiding this comment.
I reviewed this PR and didn't find any bugs. Because PosixBufferedReader sits under every Bun.spawn/child_process stdio, file stream and shell read, and the gate turns any non-EAGAIN read error into a terminal state for the reader, a human look at that trade-off would still be worthwhile.
What was reviewed:
- Flag lifecycle:
READ_FAILEDis set on the&mut selferror arm ofread_once, which is the only placeStop::Erroris produced;read,read_intoandon_pollall go throughbegin_read, so no entry can read the fd again. Nothing clears it, but every reader is started once and everyon_reader_errorparent checked (SubprocessPipeReadergoes toState::Err+deinit,FileReadercallsclose_after_error) ends the reader, so reuse/transfer viafrom()after an error does not occur. - No hang window: in
read_loopthe chunk is delivered andon_errorruns in tail position before anyis_donecheck, andread_intodispatcheson_errorimmediately, so a pull parked by the gate is always rejected in the same wakeup. - Test: traced the shim ordering (
RECV_CAP4096,EAGAIN_AT2,RECV_AT5) throughfill_scratchon a socket; #3/#4/#5 land in one call so the bytes-then-error batch is forced. As the description notes, only thechild_processcase is a deterministic guard for the fix; theBun.spawncase can pass on either build.
Extended reasoning...
The change adds one bitflag to src/io/PipeReader.rs, sets it on a fatal read error in read_once, and has begin_read refuse to read while it is set, plus two new fault-injection knobs and two test cases in test/js/bun/spawn/spawn-stdio-syscall-error.test.ts. It touches no security-sensitive surface (no auth, injection, or data exposure); the Windows reader is unaffected. The Rust diff is 12 lines and internally consistent with the existing close_if_final guard for EOF/maxBuffer, and no CODEOWNER covers the changed files. Deferring rather than approving because the reader is shared core I/O for subprocess, stream and shell paths and the fix makes every non-retry read error terminal, a behavior decision the author calls out as a downside and a maintainer should weigh.
…ntry The doc comment on READ_FAILED now says that nothing clears the bit. The tests now cover each way a consumer reaches the reader from inside the delivery of the bytes read before a failed read: - node:child_process with a 'data' listener (a pull, read_into). - node:child_process with 'readable' and read(): the reader is stopped at the highWaterMark and read() starts it again (read). - Bun.spawn with lazy: the reader starts at the first read. - Bun.spawn with the reader started at spawn. The writer waits for a line on stdin, so the first read is parked when the bytes arrive. The shim fails the recv() that follows the one that returned bytes and counts the recv() calls after it. This trigger comes from #43920. Each of the four fails without the change to begin_read, on a debug build and on a release build.
|
Updated 8:56 PM PT - Sep 24th, 2026
✅ @robobun, your commit 5f57c99f37294762f809f8eb6b9e98ff41e7d4c2 passed in 🧪 To try this PR locally: bunx bun-pr 43900That installs a local version of the PR into your bun-43900 --bun |
There was a problem hiding this comment.
I reviewed the current revision (including the second commit's four new read-entry tests) and did not find any bugs. Because this changes when the shared POSIX reader stops reading an fd, and that reader sits under Bun.spawn, node:child_process, file streams and the shell, a maintainer's look is still worthwhile — in particular on the deliberate choice that READ_FAILED is never cleared (a non-repeating errno such as ENOMEM now always ends the stream).
What was reviewed:
- Traced
READ_FAILEDthrough every fd-reading entry:read(),on_poll()andread_into()all go throughbegin_read, so a re-entrant pull from insideon_read_chunkparks instead of reading past the error; theStop::Errorarm still dispatcheson_reader_errorin tail position after the pre-error bytes are delivered. - Checked the flag survives
from(),start()andunpause(), and thatSubprocessPipeReadernever callsfrom()on an errored reader (itsState::Errarm buildsfrom_bytes_then_error), so the error cannot be dropped by the buffered-to-streaming handoff. - Checked every
on_reader_errorimplementation ends or abandons its reader (FileReaderclose_after_error, SubprocessPipeReaderdeinit, theremaining_fdscounters); none re-starts a read afterwards. - Test shim knobs are all read by the shim and reset per fd; each fixture wires the error/close events into the asserted output and the fixtures stay Linux-only behind the existing
skipIf(!isLinux || !cc).
Extended reasoning...
The diff adds a READ_FAILED bit to PosixFlags in src/io/PipeReader.rs, sets it in read_once on a non-EAGAIN errno and gates begin_read on it, plus about 200 lines of fault-injection tests in test/js/bun/spawn/spawn-stdio-syscall-error.test.ts. It touches no auth, crypto or injection surface; the sensitive part is re-entrancy in a raw-pointer read loop that runs user JS. The core Rust change is 12 lines and reads as correct, but it is a behavioral change in a shared I/O primitive used by subprocess, stream, shell and installer code, with a design choice (flag never cleared, Windows reader intentionally untouched) that a maintainer should sign off on rather than an automated approval.
Problem
test/js/bun/spawn/spawn-stdio-syscall-error.test.tsis red on alpine:"lost": -82000, not0(build 120303). node:stream: stop 'data' and 'end' after destroy() on a native-backed Readable #43739 added the case, not the bug.read_loop(src/io/PipeReader.rs:766) delivers the bytes read before a failed read, then the error. The consumer asks for more inside that delivery, and the reader reads the fd past the error. The error comes late, or never.Fix
read_oncesetsPosixFlags::READ_FAILEDon a fatal error, andbegin_readdoes not read while it is set. The request parks, thenon_reader_errorrejects it. Nothing clears the flag.uv__readclearsUV_HANDLE_READABLEbefore it reports a read error.test/js/bun/spawn/spawn-stdio-syscall-error.test.ts, 17 pass. The four new cases fail without the fix. Suites: Notes.start(), which nothing on main needs.Background
PosixBufferedReaderreads the fd behind subprocess stdio, file streams and the shell. Its parent getson_read_chunk, thenon_reader_doneoron_reader_error.FileReaderis the parent behind aReadableStream.on_read_chunkresolves the parkedpull(), and the reaction runs before it returns.FileReaderfirst needs a callback in every parent.Downsides
'end'now ends with'error', as in Node. With no listener the process stops.bun run --filterno longer drains that pipe at exit.begin_readtests one more bit.Notes
Trace without the fix (bun 1.4.3-canary.1+367d939d9, shim logs each
recv(), writerdd bs=1025). The debug build of main at 8d36bff does the same: 3 of 8 runs, two with no'error'andlost-7714150:No
'error'event: the reads reached EOF beforeon_reader_errorran, and a stored error is only returned by a later pull. When the reads park first,on_reader_errorrejects that pull and'error'comes late. That is the CI signature: the events match andlostis a negative multiple of 1025.Why alpine. In CI the failure is injected: the shim fails only the Nth
recv(), so a laterrecv()succeeds and the extra bytes show. The failingrecv()must follow bytes in the same wakeup. BusyBoxheadwrites 1025 bytes at a time, so a wakeup often holds a shortrecv()and then the failing one. coreutilsheadfills the buffer in onerecv(). The same run fails on debian when the writer is slow. With a writer that copies BusyBoxhead(stdio, 1024-byte buffer), theRECV_AT=6case fails 7 of 40 runs on bun 1.4.3-canary (release), and 36 of 40 when the writer also spins between chunks. This branch (debug): 0 of 120, and 0 of 60 withdd bs=1025.With no shim. A child with one AF_UNIX socket as fd 0 and fd 1 writes a line to stdout and reads stdin. The peer leaves that line unread, sends 8192 bytes and closes. The kernel gives the child 8192 bytes, then
ECONNRESET, then EOF.'error'ECONNRESETafter 8192 bytes'end'after 8192 bytes, no error'error'ECONNRESETafter 8192 bytesThe new cases. Each one reaches the reader from inside the delivery in a different way. Whole-file runs of the describe block:
read_intoonlychild_process,'data'listenerread_intochild_process,'readable'andread()set_flowing(true),readBun.spawn,lazyread_intoBun.spawn, reader started at spawnread_intoSPAWN_FAULT_RECV_CAP=4096makes everyrecv()short, sofill_scratchcallsrecv()again in the same wakeup.SPAWN_FAULT_RECV_EAGAIN_AT=2ends the first read loop, so the consumer's read parks and the next read is poll-driven.SPAWN_FAULT_RECV_ATthen fails after bytes in that wakeup. For the'readable'case it is 19: Copy source lines when generating error messages #3 to Finish implementing React Fast Refresh transforms #18 return 64 KiB, the highWaterMark, so the reader is stopped andread()starts it again.SPAWN_FAULT_RECV_MID_FILL=1fails therecv()that follows one that returned bytes, andSPAWN_FAULT_READS_AFTERcounts therecv()calls after it. Without the fix it is 1..stdouttook the bytes and the error. That is why this case uses state.With the fix,
CAP=4096 EAGAIN_AT=2 RECV_AT=5:Node. libuv
uv__read(src/unix/stream.c): on a read error other thanEAGAINit clearsUV_HANDLE_READABLE | UV_HANDLE_WRITABLE, callsread_cbwith the error, then stops the watcher. It callsread_cbonce for eachread(), so it never holds bytes and an error from one batch.Placement. EOF and the
maxBufferstop have the same guard at this site:close_if_finalcloses the reader before the final chunk is delivered. An error cannot use it, because a closed reader with no stored error reads as a clean end. For the same reasonREAD_FAILEDis not part ofis_done().Parents (14
BufferedReaderParentimplementations, what each does inon_reader_error):FileReader,SubprocessPipeReader,Terminal.FileResponseStream, shellsubproc.rs.filter_run.rs,multi_run.rs,lifecycle_script_runner.rs,security_scanner.rs,git_runner.rs, both cron jobs, testWorker.rs.lifecycle_script_runner.rsandcron.rsbuild a new reader withinit()for each spawn.IOReader.Only two read the same reader after an error.
filter_run.rsdrain_and_close_pipesreads once more at exit. That read is now a no-op, anddeinit()follows. Before, it could reach a second terminal callback and decrementremaining_fdstwice. The shellIOReader::start()does not restart a reader after a failed read on main, because the fired one-shot poll still counts as registered. The Windows reader gets one libuv callback for each read, so it has no bytes-then-error batch.Left open.
IOReader::start()to restart the shell's stdin reader. After this PR they must clearREAD_FAILEDthere, or acatthat follows a stdin read error gets no data, no EOF and no error.ReadState::Eofwhen the poll event carries the hangup (read_state,None if received_hup).Bun.write(file, proc.stdout)then writes 262144 of 400000 pending bytes and resolves. It is on main and in 1.4.3-canary, and this PR does not change it. It needs its own change.PosixBufferedReader::on_error. subprocess: surface stdio read and write errors instead of dropping them #41456 names it as a follow-up. Release the poll and surface the error when a Bun.file() stream read fails #41420, subprocess: surface stdio read and write errors instead of dropping them #41456 and terminal: release the wrapper after PTY EOF leaves input unflushed #42150 did it for one parent each.Suites run on the debug build:
test/js/web/streams/streams.test.js(624 pass),test/js/node/stream/node-stream.test.js(112 pass),test/js/bun/spawn/spawn-streaming-stdout.test.ts(pass),test/js/node/child_process/child_process.test.ts(81 pass, 2 fail in my container for reasons outside this change: "spawn in the default shell" reads an empty$SHELL, and "extra stdio pipes are not double-closed on GC" needs 5.0 s in a debug build against the 5 s timeout, its script printsOK). With this branch's build, the test file of #43920 passes 17 of 17 in 3 runs.#43790 is open and edits the comment above the failing case. It changes the event order in
native-readable.ts, not the reader.no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/spawn/spawn-stdio-syscall-error.test.ts