Skip to content

webcore: pin subprocess stdout FileReader while its pipe poll is live - #32743

Merged
Jarred-Sumner merged 10 commits into
mainfrom
farm/43ee931d/spawn-stdout-filereader-gc-uaf
Jun 26, 2026
Merged

Jarred-Sumner merged 10 commits into
mainfrom
farm/43ee931d/spawn-stdout-filereader-gc-uaf

Conversation

@robobun

@robobun robobun commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

What

ReadableStream::from_pipe (the proc.stdout / proc.stderr path for Bun.spawn and the shell subprocess) moves an already-registered pipe poll from the subprocess PipeReader into a freshly allocated NewSource<FileReader> and re-points the poll's owner at it. The across-read ref that keeps that box alive (waiting_for_on_reader_done + increment_count(), which upgrades this_jsvalue to Strong) was only taken in FileReader::on_start, i.e. the first time JS actually pulls from the stream.

Between from_pipe and that first pull, the poll's owner points into a box whose only ref is the JS wrapper's own Weak back-reference. If the Subprocess and its cached stdout become unreachable before anyone pulls (a fire-and-forget spawn where proc.stdout is touched but never read, and the direct child exits while something else still holds the write end), GC sweeps the JSFileInternalReadableStreamSource wrapper and frees the NewSource<FileReader> box while the poll is still armed. The next readability or EOF event dispatches into freed memory:

READ of size 8 (heap-use-after-free)
  #0 Vec::len / is_empty                                     (freed Vec<u8>)
  #2 webcore::file_reader::FileReader::on_reader_done        FileReader.rs:1008
  #3 bun_io::pipe_reader::read_socket{closure}               PipeReader.rs:846
  #4 PosixBufferedReader::read_socket                        PipeReader.rs:576
  #5 file-poll dispatch <- posix_event_loop <- us_internal_dispatch_ready_polls

freed by:   JSC::JSDestructibleObjectDestroyFunc <- MarkedBlock sweep <- MarkedSpace::sweepBlocks
allocated:  ReadableStream::from_pipe<subprocess::PipeReader> -> NewSource<FileReader>

Found by a coverage-guided GC-stress fuzzer with syscall interposition (BUN_JSC_collectContinuously=1 plus an injected EAGAIN to keep the read pending). In release builds this is silent heap corruption.

Fix

Take the across-read ref in from_pipe itself, immediately after the live reader is transferred and the JS wrapper is created, so the box is Strong-rooted for as long as the poll can fire. on_reader_done / on_reader_error release it exactly as before. FileReader::on_start now checks waiting_for_on_reader_done before taking the ref so the later handle.start() call from lazyLoadStream does not double-count on this path.

How did you verify your code works?

The test asserts the lifetime invariant directly via heapStats().objectTypeCounts.FileInternalReadableStreamSource rather than racing for the crash, since the exact UAF trigger depends on the fuzzer's syscall interposition. A detached grandchild (sh -c 'while [ ! -e FLAG ]; do sleep 0.02; done; echo x') inherits the child's stdout and keeps the write end open past the direct child's exit, so the FileReader's poll is still armed while we force GC with nothing in JS referencing the wrapper.

  • Before (git stash push -- src/ + bun bd test): duringLivePipe = 0 of 4; every wrapper swept while its poll owner still points into the freed box.
  • After: duringLivePipe >= 4; once the grandchildren exit and the pipes EOF, afterEof <= 1 (one may remain via a conservatively-rooted final Subprocess, same caveat as spawn-ipc-gc.test.ts).

Also passes spawn-streaming-stdout.test.ts, spawn-unread-stdout-gc.test.ts, spawn-ipc-gc.test.ts, spawn-stdout-iterate-leak.test.ts, and readablestream-helpers.test.ts.

ReadableStream::from_pipe moves an already-registered pipe poll from a
subprocess PipeReader into a freshly allocated NewSource<FileReader> and
re-points the poll's owner at it. The across-read ref that keeps the
NewSource box alive (waiting_for_on_reader_done + increment_count, which
upgrades this_jsvalue to Strong) was previously only taken in
FileReader::on_start, i.e. the first time JS actually pulls from the
stream. Between from_pipe and that first pull the poll's owner pointed
into a box whose only ref was the JS wrapper's own Weak back-reference.

If the Subprocess and its cached stdout became unreachable before anyone
pulled (fire-and-forget spawn where proc.stdout is touched but never
read), GC swept the JSFileInternalReadableStreamSource wrapper and freed
the box while the poll was still armed. The next readability or EOF event
then dispatched into freed memory:

  FileReader::on_reader_done          FileReader.rs:1008
    self.buffered.get().is_empty()    heap-use-after-free
  PosixBufferedReader::done           PipeReader.rs:846
  PosixBufferedReader::read_socket    PipeReader.rs:576
  file-poll dispatch <- us_internal_dispatch_ready_polls
  freed by: JSDestructibleObjectDestroyFunc <- MarkedBlock sweep

Take the across-read ref in from_pipe itself, immediately after the live
reader is transferred and the JS wrapper is created, so the box is rooted
for as long as the poll can fire. on_start now checks
waiting_for_on_reader_done so it does not double-count on the same path.

The test asserts the invariant directly via heapStats rather than racing
for the crash: with the pipe's write end held open by a detached
grandchild past the direct child's exit, every
FileInternalReadableStreamSource wrapper survives GC; once the pipe EOFs,
on_reader_done releases the ref and they become collectable.
@coderabbitai

coderabbitai Bot commented Jun 26, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

ReadableStream::from_pipe now roots active pipe-backed readers before returning, FileReader::on_start avoids duplicate hold-ref work when waiting is already set, and a new regression test checks stdout-pipe GC behavior before and after EOF.

Changes

Pipe reader lifecycle

Layer / File(s) Summary
Root pending pipe streams
src/runtime/webcore/ReadableStream.rs, src/runtime/webcore/FileReader.rs
ReadableStream::from_pipe stores the stream, checks reader completion, and roots active wrappers before returning; FileReader::on_start adds a guard so the hold-ref path does not run when waiting is already set on Unix and Windows.
Collect GC metrics
test/js/bun/spawn/spawn-stdout-filereader-gc-uaf.test.ts
A POSIX-only fixture spawns piped subprocesses, keeps stdout pipes open with detached grandchildren, samples FileInternalReadableStreamSource counts across GC phases, and emits JSON metrics.
Verify EOF collection
test/js/bun/spawn/spawn-stdout-filereader-gc-uaf.test.ts
The parent test runs the fixture under bun --smol, reaps remaining detached grandchildren, parses the emitted JSON, and asserts the live-pipe and post-EOF GC counts plus the exit code.

Possibly related PRs

  • oven-sh/bun#32582: Adjusts FileReader and pipe lifetime handling around waiting_for_on_reader_done and increment_count, which this PR further constrains and verifies with a GC regression test.

Suggested reviewers

  • Jarred-Sumner
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main change: keeping subprocess stdout FileReader pinned while its pipe poll is active.
Description check ✅ Passed The description covers what changed and how it was verified, though it uses custom headings instead of the exact template labels.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Jun 26, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 4:51 AM PT - Jun 26th, 2026

❌ @robobun, your commit 38df0f7 has some failures in Build #64933 (All Failures)


🧪   To try this PR locally:

bunx bun-pr 32743

That installs a local version of the PR into your bun-32743 executable, so you can run:

bun-32743 --bun

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/js/bun/spawn/spawn-stdout-filereader-gc-uaf.test.ts`:
- Around line 97-101: The GC/UAF regression test is too strict by asserting that
stderr is completely empty, which can be flaky under debug/ASAN. Update the test
around the spawn result handling to keep draining stderr but stop using it as
the pass/fail invariant; rely on the existing stdout and exit-code checks for
the regression signal, and only include stderr in failure diagnostics if the
test fails.
- Around line 40-43: The detached helper created in childScript can keep stdout
open forever if the fixture process crashes before the flag is written, causing
proc.stdout.text() to hang. Update the helper launched via spawn so it also
exits when the parent fixture process is gone, while still preserving the
existing flag-based completion path. Use the existing childScript setup in
spawn-stdout-filereader-gc-uaf.test.ts to add this additional termination check
so the test remains hermetic.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: c18a3846-75cf-49fe-9193-fc40167a6306

📥 Commits

Reviewing files that changed from the base of the PR and between 44c0cc4 and c3fadfc.

📒 Files selected for processing (3)
  • src/runtime/webcore/FileReader.rs
  • src/runtime/webcore/ReadableStream.rs
  • test/js/bun/spawn/spawn-stdout-filereader-gc-uaf.test.ts

Comment thread test/js/bun/spawn/spawn-stdout-filereader-gc-uaf.test.ts Outdated
Comment thread test/js/bun/spawn/spawn-stdout-filereader-gc-uaf.test.ts Outdated
robobun and others added 2 commits June 26, 2026 06:54
The detached sh helper now exits after ~30s even if the fixture crashes
before writing the flag file, and the outer test always writes the flag
in a finally block before tempDir cleanup so nothing outlives the test.

stderr is still drained but only surfaced as a diagnostic when the
fixture fails to produce JSON; the regression signal is the
duringLivePipe / afterEof / exitCode triple.
Comment thread test/js/bun/spawn/spawn-stdout-filereader-gc-uaf.test.ts Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix and test look right to me, but this is GC-lifetime/refcounting logic on the hot Bun.spawn stdout/stderr path — worth a human pass to confirm has_pending_activity() is the right gate and that there's no ref-leak path (e.g. cancel → on_reader_error with done already set).

Extended reasoning...

Overview

Two small runtime changes plus a regression test:

  • ReadableStream::from_pipe now takes the across-read ref (waiting_for_on_reader_done = true + increment_count(), upgrading this_jsvalue to Strong) immediately after transferring the live poll into the new NewSource<FileReader>, gated on reader().has_pending_activity().
  • FileReader::on_start adds && !self.waiting_for_on_reader_done.get() to both the POSIX and Windows non-lazy branches so the later handle.start() from lazyLoadStream doesn't double-count.
  • New spawn-stdout-filereader-gc-uaf.test.ts asserts the invariant via heapStats().objectTypeCounts.FileInternalReadableStreamSource: ≥ N wrappers survive GC while detached grandchildren hold the write end, then ≤ 1 after EOF.

My earlier inline comment (unbounded detached helper loop) was addressed in 171024a — the loop is now capped at 1500 iterations and the outer test writes the flag in a finally before tempDir disposal.

Security risks

None in the conventional sense (no auth, no untrusted input parsing). The change is a memory-safety fix; the risk profile is getting the refcount wrong — an under-count reproduces the UAF, an over-count leaks the NewSource<FileReader> box and pins the wrapper forever. The test's afterEof <= 1 assertion guards the leak side.

Level of scrutiny

High. This is production-critical runtime code on the path of every Bun.spawn({stdout: 'pipe'}) / proc.stdout access, and it changes when a GC root is taken/released. The diff is small (~15 net runtime lines) and the reasoning in the PR description is thorough, but refcounting + GC + event-loop-poll interactions are exactly where subtle ordering bugs hide. Specific things a human reviewer should sanity-check:

  • has_pending_activity() is the correct gate for "poll is armed" on both POSIX and Windows after reader().from(...) transfers the buffered reader.
  • The ref is always released: on_reader_done releases unconditionally when waiting_for_on_reader_done, but on_reader_error only releases when !done — if on_cancel (sets done = true) is followed by an error rather than done, the ref taken in from_pipe would not be released. This is pre-existing behavior (same flow existed when the ref was taken in on_start), but the new code makes the ref-held window start earlier and unconditionally, so it's worth a glance.
  • Ordering: to_readable_stream runs (and may throw) before the ref is taken. On the error path the poll has already been re-parented into source — also pre-existing, but worth confirming the JS wrapper's finalizer still cleans up.

Other factors

  • Bug hunter found nothing; CodeRabbit's two test-hygiene comments are resolved.
  • robobun CI shows build failures on 171024a across several build-rust/build-cpp shards; they look like infra (scripts/build/ci.ts) rather than this change, but the latest green build should be confirmed before merge.
  • Suggested reviewers (Jarred-Sumner, dylan-conway) own this area.

@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

@robobun fix:

[test/js/bun/spawn/spawn-stdout-filereader-gc-uaf.test.ts](https://github.com/oven-sh/bun/blob/3bc0f4b951d45ffc448afe304bf742161dbc7528/test/js/bun/spawn/spawn-stdout-filereader-gc-uaf.test.ts) - code 1 on :debian:[ 13 x64-asan](https://buildkite.com/bun/bun/builds/64885#019f02b7-fed5-47ee-9414-cda48b342487) (new)
bun test v1.4.0-canary.1 (3bc0f4b95)
 
113 |     }
114 |     const { iters, duringLivePipe, afterEof } = result;
115 |     // Invariant: every FileReader whose pipe poll is still armed must be
116 |     // pinned by its own Strong ref. Before the fix they were swept here
117 |     // (duringLivePipe ~ 0), which is exactly the UAF precondition.
118 |     expect(duringLivePipe).toBeGreaterThanOrEqual(iters);
                                 ^
error: expect(received).toBeGreaterThanOrEqual(expected)
 
Expected: >= 4
Received: 0
 
      at <anonymous> (/var/lib/buildkite-agent/build/test/js/bun/spawn/spawn-stdout-filereader-gc-uaf.test.ts:118:28)
 
✗ subprocess stdout FileReader is pinned while its pipe poll is live, then collectable after EOF [862.89ms]
 
 
 
0 pass
1 fail
1 expect() calls
Ran 1 test across 1 file. [1057.00ms]

@robobun

robobun commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator Author

On it. The x64-asan lane shows duringLivePipe=0, which means the gate I added in from_pipe (reader().has_pending_activity() → FilePoll::is_active() → HasIncrementedPollCount) is not matching there even though the poll was just transferred from the subprocess reader. I have switched the gate to !reader().is_done(), which is the same predicate FileReader::on_start already uses on this path, and added base / afterSpawn / streams to the fixture JSON so a repeat failure prints exactly where the count diverges. Verifying fail-before/pass-after locally now; will push shortly.

…ing_activity()

The x64-asan lane saw duringLivePipe=0 with the reader().has_pending_activity()
gate, which resolves to FilePoll::is_active() = HasIncrementedPollCount. Switch
to the same predicate FileReader::on_start already uses on this path
(!reader().is_done()): after PosixBufferedReader::from() the transferred
reader's flags contain none of IS_DONE / RECEIVED_EOF / CLOSED_WITHOUT_REPORTING
so the ref is taken unconditionally for every live pipe, and it is released by
on_reader_done / on_reader_error exactly as before.

Also fold base / afterSpawn / streams into the fixture JSON and surface the
whole result object in the failure diff so a repeat CI failure shows whether
from_pipe ran and whether the wrappers survived the first GC pass.
@robobun

robobun commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator Author

Pushed e3358d3: switched the from_pipe gate from reader().has_pending_activity() to !reader().is_done() (the same predicate FileReader::on_start already uses on this path). has_pending_activity() resolves to FilePoll::is_active() which depends on HasIncrementedPollCount; is_done() only reads the reader's own PosixFlags, which PosixBufferedReader::from() copies verbatim from the source reader, so it is reliably false for every live transferred pipe.

Verified on a local release build: the test passes 3/3 in ~130ms each. The fixture now also emits base / afterSpawn / streams in its JSON so a repeat failure would show exactly where the count diverges.

On the ref-release concern from the review: on_cancel sets done = true and then calls reader().close() → close_handle() → handle.close(Some(done_callback)) → done() → on_reader_done(), which releases the ref unconditionally when waiting_for_on_reader_done is set (it does not check done). The !done guard in on_reader_error only skips a second release when cancel already drove one via that path, so the ref taken in from_pipe is paired on every terminal path.

afterSyncGC samples the source count immediately after two synchronous
Bun.gc(true) calls with no event-loop yield, so it distinguishes 'fix did
not engage' (afterSyncGC < iters, wrappers swept synchronously) from 'pipes
reached EOF during the sleep loop' (afterSyncGC == iters but duringLivePipe
dropped afterwards). grandchildrenStarted counts touch-files written by each
detached sh so a failing lane shows whether the helpers actually came up.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/runtime/webcore/ReadableStream.rs (1)

438-445: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Condense this ownership comment to the 3-line limit.

The lifetime note is useful, but this newly added comment exceeds the repo’s 3-line maximum for code comments. As per coding guidelines, “Keep code comments to 3 lines max.”

Suggested rewrite
-        // The transferred reader already has a live poll registered with the
-        // event loop whose owner now points into this allocation. Take the
-        // across-read ref (and root the wrapper) immediately so a GC before
-        // JS first pulls cannot sweep the wrapper and free this box while the
-        // poll is still armed. `on_start` checks `waiting_for_on_reader_done`
-        // and will not take a second ref. Use the same predicate as
-        // `FileReader::on_start` so the ref is always paired with an eventual
-        // `on_reader_done`/`on_reader_error` release.
+        // The transferred live pipe poll points into this allocation, so hold
+        // the across-read ref/root before JS pulls. `on_start` sees this flag,
+        // avoiding a duplicate ref; reader-done/error releases it.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/runtime/webcore/ReadableStream.rs` around lines 438 - 445, Condense the
ownership note in ReadableStream transfer handling to fit the 3-line comment
limit while preserving the key lifetime guarantee. Keep the guidance near the
existing allocation/poll logic and mention the shared predicate with
FileReader::on_start plus the paired on_reader_done/on_reader_error release, but
remove the extra explanatory detail so the comment stays brief and actionable.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@src/runtime/webcore/ReadableStream.rs`:
- Around line 438-445: Condense the ownership note in ReadableStream transfer
handling to fit the 3-line comment limit while preserving the key lifetime
guarantee. Keep the guidance near the existing allocation/poll logic and mention
the shared predicate with FileReader::on_start plus the paired
on_reader_done/on_reader_error release, but remove the extra explanatory detail
so the comment stays brief and actionable.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 4979a52a-0b2b-4bde-a52d-dc42f3cc8630

📥 Commits

Reviewing files that changed from the base of the PR and between 3bc0f4b and e1b573c.

📒 Files selected for processing (2)
  • src/runtime/webcore/ReadableStream.rs
  • test/js/bun/spawn/spawn-stdout-filereader-gc-uaf.test.ts

robobun added 2 commits June 26, 2026 09:02
Replace the sh flag-polling loop with 'exec sleep 30' so the grandchild
cannot exit early if fractional sleep or the flag path behave differently
on a CI runner, and have the direct child linger 10ms after spawning it so
the grandchild definitely holds fd 1 before the child closes its copy. The
fixture reads each grandchild pid from the started/ touch-files and SIGTERMs
them to drive EOF; the outer test SIGKILLs any survivors in its finally.

Also condense the from_pipe ownership comment to three lines.
The CI runner sets BUN_FEATURE_FLAG_NO_ORPHANS=1 on ASAN lanes only, which
arms PR_SET_CHILD_SUBREAPER around each spawn and kills subreaper-adopted
daemons as soon as their intermediate parent exits. That is exactly the
process shape this test depends on: a detached grandchild that inherits the
direct child's stdout and holds the write end open past the child's exit.
With the flag set, the grandchildren were killed the moment the direct
children exited, the pipes EOF'd, and on_reader_done released the across-read
ref before the fixture could observe it, which is why only the x64-asan lane
saw duringLivePipe=0 while debug/release/release-asan all passed locally.

The test already enforces the invariant NO_ORPHANS exists for: the fixture
SIGTERMs every grandchild by pid and the outer test SIGKILLs survivors in a
finally block, so nothing can outlive it.
@robobun

robobun commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator Author

Root-caused the x64-asan-only failure: the CI runner sets BUN_FEATURE_FLAG_NO_ORPHANS=1 only on ASAN lanes (to stop orphaned test subprocesses piling up and OOM-killing agents). That flag arms PR_SET_CHILD_SUBREAPER and kills subreaper-adopted daemons the moment their intermediate parent exits, which is exactly the process shape this test relies on: a detached grandchild inheriting the direct child's stdout and holding the write end open past the child's exit.

So on the asan lane the grandchildren were killed as soon as each direct child exited, every pipe EOF'd, and on_reader_done released the across-read ref before the fixture could observe it. That is why it reproduced on only that lane while debug, release, and release-asan all passed locally. Setting the flag locally reproduced CI byte for byte: duringLivePipe=0, afterSyncGC=1, grandchildrenStarted=5.

The fix is in the test (d7c5b0b): clear BUN_FEATURE_FLAG_NO_ORPHANS on the fixture spawn. The test already enforces the invariant that flag exists for, explicitly: the fixture SIGTERMs every grandchild by pid, and the outer test SIGKILLs any survivor in a finally. Verified the full 2x2 locally: the test fails with and without the flag when src/ is reverted to main, and passes with and without the flag with the fix.

The runtime fix itself (from_pipe taking the across-read ref) was never the problem; afterSyncGC only ever dropped below 4 because the pipes were already at EOF.

Comment thread test/js/bun/spawn/spawn-stdout-filereader-gc-uaf.test.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/js/bun/spawn/spawn-stdout-filereader-gc-uaf.test.ts`:
- Around line 49-51: The test currently relies on a fixed 10ms delay in the
intermediate shell setup, which does not guarantee the detached child has
created its pid file before the directory snapshot is taken. Update the spawn
test flow around the setup used by the `spawn`/`setTimeout(..., 10)` step and
the later pid-file snapshot logic to wait for the actual pid file condition
instead of sleeping, so the test proceeds only after the detached shell has
written its pid file and stdout can close deterministically.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 48ea0918-7dd4-476e-9226-05c63732281f

📥 Commits

Reviewing files that changed from the base of the PR and between e1b573c and d7c5b0b.

📒 Files selected for processing (2)
  • src/runtime/webcore/ReadableStream.rs
  • test/js/bun/spawn/spawn-stdout-filereader-gc-uaf.test.ts

Comment thread test/js/bun/spawn/spawn-stdout-filereader-gc-uaf.test.ts Outdated
robobun added 2 commits June 26, 2026 09:43
Replace the 10ms setTimeout with a bounded poll for the grandchild's pid
file, so the direct child only exits once the grandchild is provably past
posix_spawn (owning fd 1) and its pid is recorded. await proc.exited then
implies the fixture's readdirSync sees every grandchild, making the kill
loop deterministic instead of racing a timer.
@robobun

robobun commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator Author

CI update: on the latest push (26383ab) the test passes on the x64-asan lane that had been failing every build (1 pass, 0 fail on the [3/224] shard of build 64926), so the BUN_FEATURE_FLAG_NO_ORPHANS root cause and fix are confirmed end to end.

The only hard-failed job in that build is darwin 26 aarch64 - test-bun, which died on buildkite-agent artifact download timed out after 120s before running any tests, the second build in a row on the same macOS agent. The darwin-aarch64-build-bun artifact was produced successfully and other darwin shards downloaded it, so this is agent-local infra, not the diff. I've pushed one ci: retrigger to re-roll it.

If the darwin artifact download flakes again, the diff is ready: the runtime change is small (root the NewSource<FileReader> in from_pipe so the box cannot be GC'd while its transferred pipe poll is still armed), the gate's fail-before/pass-after holds, and every lane that actually exercises the change is green.

@robobun

robobun commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator Author

This is ready for review; the remaining CI red is infrastructure, not the diff.

Across the last three builds (64926, 64933, and the ci: retrigger re-roll) the only hard-failed job is :darwin: 26 aarch64 - test-bun, and in every one of them it fails before running a single test:

Error: buildkite-agent artifact download timed out after 120s for step 'darwin-aarch64-build-bun'.
Refusing to continue with a partial download (would silently fall back to the wrong binary).

The darwin-aarch64-build-bun artifact is produced successfully every time and other darwin shards download it fine, so this is one macOS agent's artifact download, not anything in this PR. I've used my one retrigger.

Everything that actually exercises the change is green:

  • The regression test passes on the x64-asan lane (1 pass, 0 fail), the only lane where it ever failed, now that the test clears BUN_FEATURE_FLAG_NO_ORPHANS (the ASAN-lane-only env var that was killing the detached grandchild and EOF'ing the pipe before the fixture could measure).
  • The runtime fix itself is unchanged since e3358d3 and is small: ReadableStream::from_pipe takes the across-read ref (Strong-rooting the FileInternalReadableStreamSource wrapper) the moment it re-parents the subprocess's live pipe poll into the new NewSource<FileReader>, instead of waiting for the first JS pull in FileReader::on_start. Without it the box holding the poll's owner pointer can be swept while the poll is still armed, and the next readability/EOF event dispatches into freed memory (FileReader::on_reader_done via PosixBufferedReader::read_socket).
  • Fail-before / pass-after is verified locally on debug, release, and release-asan: with src/ reverted to main, duringLivePipe = 0 of 4; with the fix, 4 of 4.

All review threads (claude and CodeRabbit) are addressed and resolved.

@Jarred-Sumner
Jarred-Sumner merged commit 3f464e8 into main Jun 26, 2026
77 of 78 checks passed
@Jarred-Sumner
Jarred-Sumner deleted the farm/43ee931d/spawn-stdout-filereader-gc-uaf branch June 26, 2026 20:54
Jarred-Sumner pushed a commit that referenced this pull request Jun 28, 2026
… and on_reader_error (#32921)

### What

`Bun.spawn` stdout/stderr pipe readers still hit a heap-use-after-free
on current main under fault injection, with the faulting pc symbolizing
to `PosixBufferedReader::on_error` and
`PosixBufferedReader::register_poll`. #32743 fixed the read-completion
path of this class (`on_reader_done` reached from `read_socket`) but the
error and poll-registration completion paths have the same lifetime
defect.

### Cause

`PosixBufferedReader::read_with_fn` holds `&mut` into the
`NewSource<FileReader>` box for the whole epoll dispatch. Mid-loop,
`FileReader::on_read_chunk` resolves the pending `read()` promise via
`p.run()`, which enters JS and drains microtasks. If user JS calls
`reader.cancel()` there, the chain

```
cancel -> on_cancel -> reader.close() -> done() -> on_reader_done()
```

runs synchronously and releases the across-read ref that #32743 added.
That drops the box's count to the JS wrapper's own ref and downgrades
the wrapper from Strong to Weak. A GC inside that same microtask drain
sweeps the wrapper, and its finalizer frees the box out from under
`read_with_fn`. On the next inner-loop iteration the (now closed) fd
returns `EBADF` and read_with_fn calls `parent.on_error(err)` on the
freed reader, or `parent.register_poll()` on the retry arm, reproducing
the reported frames. `FileReader::on_reader_error` has the identical
shape via its own `p.run()` followed by reads of `self`.

Both callbacks are reachable from `Bun.spawn` and from
`Bun.file(fd).stream()`, so the blast radius is any app that streams a
subprocess pipe.

### Fix

Hold one additional ref on the `NewSource` box across `p.run()` at both
call sites (`on_read_chunk` and `on_reader_error`), released at true
tail. The re-entrant release from `on_reader_done` then lands at a count
of two instead of one, so it never crosses the downgrade threshold and
the wrapper stays Strong-rooted for the remainder of the dispatch. The
box is collected normally on a later, off-stack GC.

`on_read_chunk` also now returns `false` when the re-entrant cancel
marked the reader done. `read_with_fn`'s two flush sites consult that
return only when `received_hup` is false, so this stops the non-HUP path
from issuing another `recv` on the cancelled reader's closed fd (the
source of the spurious `EBADF` that previously reached `on_error`). The
HUP-path flushes intentionally ignore it (`&& !received_hup` is a
documented hang fix for shell blocking pipes), so they still loop to
EOF; that is pre-existing, benign, and left alone, because the pin and
not the return value is what makes both branches memory safe, and
tightening it means changing `PipeReader.rs` infrastructure shared by
every buffered-reader parent.

The sibling Posix site at `PipeReader::start` (SubprocessPipeReader.rs)
already holds a `ScopedRef` across the same re-entrancy and is
unaffected.

### Verification

`test/js/bun/spawn/spawn-stdout-filereader-gc-uaf.test.ts` gains a
second test next to the #32743 one. A detached grandchild saturates the
stdout socketpair while the parent is blocked in `sleepSync`, so the
payload arrives in one poll dispatch and `p.run()` fires with
`read_with_fn` still on the stack. Which of `read_with_fn`'s two flush
sites delivers it depends on the kernel socketpair buffer (the 128 KiB
mid-loop flush on Linux, the retry flush on macOS, whose default is a
few KiB); the pin guards both identically, so the test does not depend
on a platform-specific buffer size. The `.then` callback cancels the
reader and samples
`heapStats().protectedObjectTypeCounts.FileInternalReadableStreamSource`
immediately after.

That counter observes the `JsRef` Strong directly, so the test asserts
the lifetime invariant rather than racing a GC into the vulnerable
window, and is deterministic in both directions (no ASan or
`collectContinuously` dependence):

- without the `src/` change: `protectedAfter` is `0` (Strong dropped
mid-dispatch, the UAF precondition) and the test fails on that assertion
with both preconditions green
- with it: `protectedAfter` is `1` and both tests in the file pass

The `on_reader_error` pin is the same bracket at the sibling site named
by the reported `register_poll` frame, whose failure path calls it. It
is not separately tested, and that is a proven limit rather than an
untried one. Through the only live `Pending` consumer (the pull
promise), the window cannot be reached from JS: the native pull
promise's rejection arm in `#pull` errors the stream before any user
reaction runs, and `readableStreamCancel` is a no-op on an errored
stream, so a `reader.cancel()` inside a rejection handler never
re-enters `on_reader_done`. I verified this empirically with the same
dup2 fd swap `spawn-pipe-read-error-leak.test.ts` uses to force a
non-retry `recv` error: `on_reader_error` fires, the in-handler
`cancel()` produces no `onReaderDone` and no change in the protected
count on either build, so no assertion through that route distinguishes
fixed from unfixed. The `PendingFuture::Handler` consumer, the only
other route to `p.run()` there, has no installers. The pin stays because
`on_reader_error` reads `self` after a call that runs user JS; that
contract holds today only through the non-local ordering inside
`#pull`'s JS, which nothing enforces, and the bracket is balanced and
free.

Also re-ran `spawn-pipe-read-error-leak`, `spawn-streaming-stdout`,
`spawn-unread-stdout-gc`, `spawn-stdout-iterate-leak`,
`readablestream-helpers`, `spawn-ipc-gc`, and
`native-source-onclose-leak`: all green.
Jarred-Sumner pushed a commit that referenced this pull request Jul 4, 2026
### What

`FileReader::on_reader_done` has the same shape #32921 fixed in its two
siblings: it runs user JavaScript and then reads `self` with no refcount
pin. A heap-use-after-free was reported there under fault injection on
an instrumented build (READ in `FileReader::on_reader_done`, on the
`self.buffered` length read that follows `p.run()`). #32921 added the
pin to `on_read_chunk` and `on_reader_error` but not to
`on_reader_done`, one screen below them in the same file.

### Cause

`on_reader_done` resolves the pending pull with `Done`/`OwnedAndDone`
via `p.run()`, which fulfills a JS promise and drains microtasks. After
that returns, it still reads `self.buffered`, calls
`(*self.parent()).on_close()` (which can invoke a native consumer's
`close_handler`), and reads `self.waiting_for_on_reader_done`. `self` is
a field of a heap-allocated, refcounted `NewSource<FileReader>`; nothing
holds an extra reference across those calls, so any release that lands
inside them frees the box out from under the rest of the function.

On current `main` the function is safe only through a non-local
invariant: while `p.run()` is executing, the count is 2 (the JS
wrapper's ref plus the across-read ref that `on_reader_done` itself
releases at its tail), and no synchronous release is reachable from JS
inside that window, because the re-entrant `cancel -> on_cancel ->
reader().close() -> done()` chain that #32921's test drives is gated on
`!self.reader().is_done()`, which is always false once `on_reader_done`
is on the stack. That is the same situation #32921 described for its
`on_reader_error` pin: the contract holds today only through non-local
ordering that nothing enforces.

### Fix

Hold one additional ref on the `NewSource` box for the duration of
`on_reader_done`, released at true tail, matching the bracket
`on_read_chunk` and `on_reader_error` already have. The across-read
release then lands at a count of two instead of one, the wrapper stays
Strong-rooted for the rest of the dispatch, and the box is collected
normally on a later, off-stack GC.

I audited the other `Pending::run` call sites in `src/runtime/webcore`.
`ByteStream::on_data`'s is tail-positioned, `ByteStream::on_cancel`'s is
entered from JS with the wrapper a conservative stack root, and
`ByteStream::finalize` defers any JS-visible resolution to the next
tick. `on_reader_done` was the only remaining run-JS-then-read-`self`
site without a pin.

### Verification

`test/js/bun/spawn/spawn-stdout-filereader-gc-uaf.test.ts` (the file
holding the #32743 and #32921 tests) gains a third test for the third
member of the family. `cat` holds the pipe open until its stdin EOFs, so
`reader.read()` is armed before the pipe can close; `stdin.end()` then
drives `on_reader_done` with the pull Pending and no fixed sleeps. The
`{done: true}` handler, which runs synchronously inside
`on_reader_done`'s `p.run()` microtask drain, asserts the source stays
Strong-protected through the most aggressive teardown reachable from
there (`cancel()` plus dropping every JS reference plus `Bun.gc(true)`).

Being straight about what that test is: it passes on the unfixed build
too, because the across-read ref carries the invariant today (see
Cause). It is a guard on the invariant the pin formalizes, and the
canary for the day a change makes the release reachable from inside
`p.run()`, which is exactly what happened to `on_read_chunk` between
#32743 and #32921. I could not construct a plain-JS input that
distinguishes the pin. In a 215-iteration probe on the unfixed ASan
debug build across five shapes (pipe EOF with a pending read plus an
in-handler `cancel()`, the chunk-then-done variant, `node:child_process`
`end` and `close` handlers, and a cancel-initiated `on_reader_done`),
with `Malloc=1` and `BUN_JSC_collectContinuously=1`, ASan never fired
and the `FileInternalReadableStreamSource` count never dropped inside
the window.

Also re-ran `spawn-streaming-stdout`, `spawn-unread-stdout-gc`,
`spawn-stdout-iterate-leak`, `spawn-pipe-read-error-leak`,
`spawn-ipc-gc`, and `native-source-onclose-leak` on the debug (ASan plus
debug_assert) build: all green, which also checks the bracket's refcount
balance on every path those exercise.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants