Skip to content

streams: throw when a read request, pull-into or byte queue deque cannot grow - #42659

Merged
Jarred-Sumner merged 3 commits into
robobun/337717e9/streams-growth-abortfrom
robobun/1d6807ba/streams-request-queues
Sep 14, 2026
Merged

Jarred-Sumner merged 3 commits into
robobun/337717e9/streams-growth-abortfrom
robobun/1d6807ba/streams-request-queues

Conversation

@robobun

@robobun robobun commented Sep 13, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #42649. It reuses the maxVectorSize() helper and the 64 KiB synthetic limit harness from that PR, so it is opened against that branch. It covers the streams rows of #42648. The microtask queue row is out of scope.

Problem

  • A pending reader.read(), a pending read(view), and a byte controller.enqueue() each add one entry to a WTF::Deque. When a deque cannot grow, Deque::expandCapacity calls CRASH() and the process exits with panic(main thread): abort() called (SIGABRT, 134). Script cannot catch it.
  • The appends are m_readRequests and m_readIntoRequests (ReadableStreamOperations.cpp:203, :213), m_pendingPullIntos (JSReadableByteStreamController.cpp:404, :1018, :1045) and the byte queue (:770). The limits are 2^27 - 1 requests or 2^26 - 1 chunks, at 8 GB or more of RSS.

Fix

  • Hoist the deque bound from StreamQueue.h into VectorSizeLimit.h as Bun::maxDequeSize<T>(). The value queue in StreamQueue.h uses it too.
  • readableStreamAddReadRequest and readableStreamAddReadIntoRequest take a JSGlobalObject* and throw RangeError: Out of memory before they take the cell lock. The six callers propagate the exception, so read() rejects and the stream stays readable. The byte controller checks the pull-into deque before it transfers the caller's buffer, and adds the request before the descriptor, so a refused read leaves nothing behind.
  • readableByteStreamControllerEnqueueChunkToQueue errors the stream and throws when the byte queue is full. The chunk's buffer is already transferred at that point, so a silent drop is not an option. This is what the same function does for a failed clone.
  • Verified: test/js/web/streams/streams.test.js, four new tests in the synthetic limit block. Each fails on the base branch and passes with the fix. Also the rest of test/js/web/streams/, test/js/web/fetch/body-stream.test.ts, test/js/node/stream/node-stream.test.js and the spawn stdin stream tests.

Background

  • A WTF::Deque is a ring buffer over a WTF::Vector. Its capacity is a power of two and one slot stays empty, so the most entries it holds is bit_floor(maxVectorSize) - 1. A Vector refuses a capacity over 2^31 - 1 bytes with CRASH(), not with a false return.
  • A web stream reader keeps each pending read in [[readRequests]] (default reader) or [[readIntoRequests]] (BYOB reader). A byte stream also keeps a [[pendingPullIntos]] entry per BYOB read, and per default read when autoAllocateChunkSize is set.
  • BUN_FEATURE_FLAG_SYNTHETIC_MEMORY_LIMIT lowers maxVectorSize() in a child process. The tests set it to 64 KiB, which brings the bounds down to 8191 reads and 2047 chunks.
  • cellLock() guards the barrier containers against a concurrent GC visit. A throw allocates, so every check runs before the lock is taken.
Notes
  • On the base branch the new tests fail fast on a mismatch, not on a timeout: a refused read never rejects, and the byte enqueue loop never throws. Without the synthetic limit the unfixed build aborts at the real bounds from the issue table.
  • m_writeRequests (WritableStreamOperations.cpp:264) is not changed. The value queue of the same stream fills first, and streams: throw instead of aborting when a script-sized container cannot grow #42649 bounds that.
  • readableStreamAddReadRequest is reached from pullSteps of both controllers, the direct controller's onPull, and the ControllerKind::None and Direct arms of readableStreamDefaultReaderRead. Each of those already sits under a throw scope whose callers check for an exception, because pullSteps could already throw through the pull algorithm.
  • The spec order in pullSteps and PullInto is "append the descriptor, then add the request". The PR adds the request first. No JS runs between the two, and nothing reads either deque in between.
  • test/js/web/streams/streams-leak.test.ts has one test that times out under the debug build on the base branch as well (5000 pipe rounds in 5 s).

no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/web/streams/streams.test.js

…not grow

A reader keeps one deque entry per pending read() or read(view), a byte
stream keeps one pull-into descriptor per pending read, and the byte
controller keeps one queue entry per enqueue(). Each of these is a
WTF::Deque, and Deque::append calls CRASH() when the next capacity is
not valid for a Vector. A script could reach that with 2^26 or 2^27
entries and abort the process.

Hoist the Deque bound from StreamQueue.h into VectorSizeLimit.h as
Bun::maxDequeSize<T>(), and check it before each append:

- readableStreamAddReadRequest and readableStreamAddReadIntoRequest
  take a JSGlobalObject and throw an out-of-memory RangeError. Their
  callers propagate the exception, so read() rejects and the stream
  stays readable.
- The byte controller checks the pull-into deque before it transfers
  the caller's buffer, and adds the read request before the pull-into
  descriptor, so a refused read leaves nothing behind.
- readableByteStreamControllerEnqueueChunkToQueue errors the stream and
  throws when the byte queue is full, as a failed clone does, because
  the chunk's buffer is already transferred at that point.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. Because it reorders spec-step append order in readableByteStreamControllerPullInto/pullSteps and threads new fallible paths through the byte controller, a human look at the streams-spec conformance side would still be worthwhile.

What was reviewed:

  • Exception-check discipline: every newly-fallible call site (readableStreamAddReadRequest/AddReadIntoRequest/EnqueueChunkToQueue) has RETURN_IF_EXCEPTION or RELEASE_AND_RETURN; throws happen before cellLock() is taken.
  • Orphan-state audit: refused reads leave no pull-into descriptor (append reordered after the request-deque check); pendingPullIntosFull() runs before transferArrayBufferImpl so a refused BYOB read keeps the caller's buffer; the two 8-byte deques share the same bound so the pre-transfer check covers both.
  • All call sites of the three re-signatured helpers grepped and updated; EnqueueDetachedPullIntoToQueue already had RETURN_IF_EXCEPTION after the cloned-chunk call.
  • Test arithmetic matches maxDequeSize<T>() for the 64 KiB synthetic limit (8191 reads, 2047 byte-queue entries).
Extended reasoning...

Overview

This PR extends the deque-overflow hardening from the stacked base (#42649) to four more WTF::Deque sites in the C++ ReadableStream implementation: the default/BYOB reader's pending-read deques, the byte controller's m_pendingPullIntos, and the byte controller's chunk queue. maxDequeSize<T>() is hoisted from StreamQueue into VectorSizeLimit.h; readableStreamAddReadRequest/AddReadIntoRequest and readableByteStreamControllerEnqueueChunkToQueue gain a JSGlobalObject* and throw OutOfMemoryError when the target deque is full. All six call sites of AddReadRequest (default/byte/direct controllers, default reader) and all four of EnqueueChunkToQueue are updated to propagate. In the byte controller, the pull-into-descriptor append is reordered after the (now-fallible) read-request append so a refused read leaves no orphan, and pendingPullIntosFull() guards run before any buffer transfer. Four test.concurrent subprocess tests exercise each deque under the 64 KiB synthetic limit.

Security risks

No new attack surface. The change converts script-reachable CRASH() (SIGABRT DoS) into catchable RangeError: Out of memory, which is strictly a hardening improvement. The throws happen before any cellLock() is taken, so no allocation-under-lock. The m_pendingPullIntos and m_readIntoRequests deques have identical 8-byte element sizes and grow in lockstep per BYOB read, so the pre-transfer pendingPullIntosFull() check also bounds the post-transfer AddReadIntoRequest call — a refused read cannot detach the caller's buffer and then fail. readableByteStreamControllerError on the full-byte-queue path clears m_pendingPullIntos and resets the queue before rethrowing, so no partial state survives.

Level of scrutiny

High. JSC bindings C++ is REVIEW.md's "most-blocked category" for memory safety, and this PR deliberately deviates from WHATWG spec step order ("append descriptor, then add request" → "add request, then append descriptor") in pullSteps and both branches of PullInto. The PR's argument that nothing observes the order looks correct — no user JS runs between the two appends, and no code reads either deque in between — but spec-order deviations in Web-standard APIs are exactly the kind of thing a maintainer familiar with the streams WPT suite should sign off on.

Other factors

The exception-check hygiene is clean: DECLARE_THROW_SCOPE at function top, RETURN_IF_EXCEPTION after each newly-fallible call, RELEASE_AND_RETURN on tail calls, [[unlikely]] on the full-deque branches. I grepped for every call site of the three re-signatured helpers and confirmed all are updated. readableByteStreamControllerEnqueueDetachedPullIntoToQueue already had RETURN_IF_EXCEPTION after EnqueueClonedChunkToQueue, so its transitive path is covered. Tests reuse the base branch's runInSubprocess/describeError/outOfMemory harness, use test.concurrent, assert a combined {stdout, stderr, exitCode} object with stdout/stderr before exitCode, and the deque-size arithmetic (8191 reads at 8 B/slot, 2047 chunks at 24 B/entry) matches std::bit_floor(maxVectorSize<T>()) - 1 for the 64 KiB limit. StreamQueue::prepend is not guarded, but its only caller re-prepends a just-removed head entry so the queue never grows past its prior size.

@robobun

robobun commented Sep 13, 2026

Copy link
Copy Markdown
Collaborator Author

On the append order in pullSteps and PullInto: the spec appends the pull-into descriptor and then adds the read request as two consecutive "!" steps. No algorithm runs between them, and nothing reads [[pendingPullIntos]] or [[readRequests]] until the caller returns. So swapping them is not observable, and it means a refused request never leaves a descriptor behind. The WPT-derived cases in test/js/web/streams/streams.test.js pass with the swap.

…e is full

A tee gives every chunk to both branches. When the unread branch's
queue cannot take the next chunk, the enqueue throws out of the tee's
chunk steps microtask: the error is uncaught, m_reading stays true, and
the other branch hangs. End the tee as a clone failure does: error both
branches and cancel the source with the error. The default tee and both
byte tee paths share one helper for this.

A direct stream runs pull() for each read. When the request deque
refused the read after pull() ran, a close() or flush that pull()
deferred was dropped with it. Check the bound before pull() runs.
Comment thread src/jsc/bindings/VectorSizeLimit.h Outdated
Comment thread src/jsc/bindings/webcore/streams/JSDirectStreamController.cpp Outdated
Comment thread src/jsc/bindings/webcore/streams/JSReadableByteStreamController.cpp Outdated
Comment thread src/jsc/bindings/webcore/streams/JSReadableByteStreamController.cpp Outdated
Comment thread src/jsc/bindings/webcore/streams/JSReadableByteStreamController.cpp Outdated
Comment thread src/jsc/bindings/webcore/streams/JSReadableByteStreamController.cpp Outdated
Comment thread src/jsc/bindings/webcore/streams/ReadableStreamOperations.cpp Outdated
Comment thread src/jsc/bindings/webcore/streams/ReadableStreamOperations.cpp Outdated
@robobun

robobun commented Sep 13, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 4:19 PM PT - Sep 13th, 2026

✅ Your commit a988213 has passed in Build #115317! 🎉


🧪   To try this PR locally:

bunx bun-pr 42659

That installs a local version of the PR into your bun-42659 executable, so you can run:

bun-42659 --bun

Comment thread src/jsc/bindings/webcore/streams/ReadableStreamOperations.cpp Outdated
Comment thread src/jsc/bindings/webcore/streams/StreamQueue.h Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

@robobun

robobun commented Sep 13, 2026

Copy link
Copy Markdown
Collaborator Author

CI is green on a988213 and the review threads are resolved. This PR is ready for review once #42649 lands. It is opened against that branch, so GitHub retargets it to main when #42649 merges.

@Jarred-Sumner
Jarred-Sumner merged commit 720791b into robobun/337717e9/streams-growth-abort Sep 14, 2026
6 checks passed
@Jarred-Sumner
Jarred-Sumner deleted the robobun/1d6807ba/streams-request-queues branch September 14, 2026 02:22
Jarred-Sumner added a commit that referenced this pull request Sep 14, 2026
…ot grow (#42649)

### Problem
- Eight containers in the C++ web streams grow with script input and
call `CRASH()` when full: `panic(main thread): abort() called`, exit
134. Every release since #33193 aborts, and Bun 1.3.14 does not. Fuzzing
found them, with no user report.
- Four are cheap: `StreamQueue::enqueueValueWithSize`
(`StreamQueue.h:138`, the 67,108,864th `enqueue()`, 1.1 GB),
`BunTextAccumulator::pieces` (the 262,343,954th binary `write()` under
`.text()`, 2.1 GB), `streamingUTF8Decode` (`WebStreamsMisc.cpp:101`) and
`concatenateChunks` (`BunStreamConsumers.cpp:402`), each with one
2^31-byte input.
- Four need 8 GB or more: the read request, read-into and pull-into
`Deque`s and the byte queue (from #42659, merged into this branch).

### Fix
- Each site throws `RangeError: Out of memory`, as #37237 does. `Deque`
has no fallible append, so those sites check `Bun::maxDequeSize<T>()`
before the cell lock. The `Vector` sites call `tryAppend`, `tryGrow` or
`tryReserve*`.
- A failed `enqueue()` errors the stream. A refused `read()` rejects and
the stream stays readable. `writer.write()` never throws: the stream
errors and the pending writes reject. A full `tee()` branch errors both
branches and cancels the source.
- In `concatenateChunks` this PR only stops the abort. #37239 (open)
makes totals up to 4 GiB resolve again.
- Verified: 14 new tests in `streams.test.js`, 2 in
`streams-string-limit.test.ts`. All 16 fail on the `main` sources.
Self-reviewed: 14 concerns raised, 12 addressed. The other 2 ask for a
source lint (#42648).

### Background
- `WTF::Vector<T>` holds `INT32_MAX / sizeof(T)` elements. `append`
calls `CRASH()` past that, and `tryAppend` returns false. `WTF::Deque`
has a power-of-two capacity and keeps one slot empty.
- `Bun::maxVectorSize<T>()` and `Bun::maxDequeSize<T>()` are those
bounds, lowered by `Bun__stringSyntheticAllocationLimit`. The tests set
it to 64 KiB in a child process.
- `cellLock()` guards GC references against the concurrent marker. A
throw allocates from the GC heap, so it happens outside the lock.

<details><summary>Notes</summary>

**Repros, on a release build of `main` (f04caca) and on Bun 1.3.14**

```js
// 1. main: exit 134 after 2.6 s at 1.11 GB. 1.3.14: finishes, 4.7 GB.
let c;
new ReadableStream({ start(ctrl) { c = ctrl; } }, { highWaterMark: Infinity });
for (let i = 0; i < 67108864; i++) c.enqueue(1);

// 2. main: exit 134 after 10 s at 2.13 GB. 1.3.14: resolves, 4.4 GB.
const chunk = new Uint8Array(1);
await new ReadableStream({
  type: "direct",
  pull(c) { for (let i = 0; i < 262343954; i++) c.write(chunk); c.end(); },
}).text();

// 3. main: exit 134 at once, 29 MB. 1.3.14: exits 0.
const w = new TextDecoderStream().writable.getWriter();
w.write(new Uint8Array([0xe2]));
w.write(new Uint8Array(2 ** 31));

// 4. main: exit 134 at once, 32 MB. 1.3.14: resolves with 2,200,000,001 bytes.
const big = new Uint8Array(1100000000);
await Bun.readableStreamToArrayBuffer(new ReadableStream({
  start(c) { c.enqueue("a"); c.enqueue(big); c.enqueue(big); c.close(); },
}));
```

**Provenance**
- Sites 1 to 3 come from one fuzzing census of `main`. Site 4 is the
abort #37239 reported on 2026-08-09. A review of this diff found that it
was still open in a file this PR edits.
- The four expensive `Deque`s come from the same census. This PR first
left them out and listed them in #42648. #42659 then fixed them on top
of this branch, and it was merged into it.
- All eight containers first appear in #33193, the C++ rewrite of the
streams. Before it the queues were JS arrays, which throw `RangeError:
Out of memory`.
- This is hygiene under the `REVIEW.md` rule that a failure user input
can reach is a catchable error. It is not urgent.

**With this PR, on a release build, at the real limits** (measured
before #42659 was merged in)
- Repro 1: `enqueue()` throws at 67,108,862 values. Exit 0, 2.9 s, 1.08
GB.
- Repro 2: `write()` throws at 262,343,953 pieces, then `text()`
resolves with that many characters. Exit 0, 12.1 s, 2.67 GB peak.
- Repro 3: the second `write()` rejects. Exit 0, 0.1 s, 20 MB.
- A `WritableStream` whose queue holds exactly 67,108,862 writes closes
and drains (5.7 GB). One more write errors the stream: `writer.closed`
and all 67,108,863 write promises reject with the `RangeError`, and no
chunk reaches the sink (3.75 GB).
- That last run gave the write promises no reactions. With a reaction on
each, `writableStreamFinishErroring` queues 67 million jobs in one loop,
and JSC's microtask queue aborts at the 33,554,432nd pending job. A
plain `Promise.resolve().then()` loop does the same on `main` and on
1.3.14. #42648 tracks it.
- The four expensive `Deque`s were not run at their real limits (8.5 GB
to 42 GB). The synthetic limit covers them.

**Where each number comes from**
- A value queue entry is 16 bytes, so the largest legal `Deque` capacity
is 2^26. A ring of capacity C holds C - 1 entries: 67,108,863. After
this change that is 67,108,862 values plus the slot for the
`WritableStream` close sentinel. A queue takes one sentinel at most,
because `close()` rejects when a close is already queued.
- A piece is 8 bytes. `Vector::append` grows by a quarter, and the step
from 262,343,953 passes the largest capacity (268,435,455). `tryAppend`
refuses at the same point. `maxVectorSize` binds only when a test lowers
the limit.
- `concatenateChunks` also keeps a 16-byte entry per chunk. It now
reserves them in one fallible step.
- A read request, a read-into request and a pull-into descriptor each
take an 8-byte slot: 134,217,727 entries. A byte queue entry is 24
bytes: 67,108,863 entries.

**What each refused operation leaves behind**
- The byte controller checks the pull-into `Deque` before it transfers
the caller's buffer, and it adds the request before the descriptor. A
refused `read(view)` keeps its view and leaves no descriptor.
- `readableByteStreamControllerEnqueueChunkToQueue` errors the stream
when the byte queue is full. The buffer of the chunk is already
transferred at that point, so the chunk cannot be handed back. The same
function does this for a failed clone.
- A direct stream refuses the read before `pull()` runs, so a `close()`
that `pull()` defers is not lost.
- `teeAbortWithError` is the clone-failure path of the byte tee, moved
into one function. The default tee and the byte tee now use it when a
branch enqueue throws.

**Sites left as on `main`**
- The write request `Deque` (`WritableStreamOperations.cpp:264`). The
value queue of the same stream fills first, and `enqueueValueWithSize`
bounds that.
- JSC's microtask queue (#42648).
- `StreamQueue::prepend` has no caller.

**`VectorSizeLimit.h`**
- It started as the file #42214, #40577 and #42220 add, byte for byte.
#42659 added `maxDequeSize` and `#include <bit>` to it. Whichever of
those PRs lands after this one takes this copy, which is a superset.

**The tests**
- Fourteen tests run children with the 64 KiB limit, the knob
`scrypt.test.ts`, `argon2.test.ts` and #42214 use. On the `main` sources
every loop runs to its end and reports no error, or a refused read never
rejects. So the fail-before is an assertion diff, not an abort, and no
test runs into its timeout.
- The `TextDecoderStream` test and the `arrayBuffer()` byte-total test
need the real sizes: a later check in the same function has the same
threshold as the synthetic limit. Each child reserves its big chunk and
never touches it. On the `main` sources each child exits 134.
- The `WritableStream` test also closes a writer whose queue is exactly
full, to cover the sentinel slot.
- `streams-leak.test.ts` fails one RSS threshold on a debug ASAN build
in this container, with and without this diff (761 MB against a 700 MB
bound on the `main` sources).
</details>

<!-- robobun:evidence:begin -->

---

**no test proof** · iteration 0 · platform-specific test(s) that do not
run on this machine, deferring to CI, which covers all platforms:
test/js/web/streams/streams.test.js

<!-- robobun:evidence:end -->

---------

Co-authored-by: robobun <robobun@bun.sh>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
usrbinkat pushed a commit to usrbinkat/bun that referenced this pull request Sep 15, 2026
…ot grow (oven-sh#42649)

### Problem
- Eight containers in the C++ web streams grow with script input and
call `CRASH()` when full: `panic(main thread): abort() called`, exit
134. Every release since oven-sh#33193 aborts, and Bun 1.3.14 does not. Fuzzing
found them, with no user report.
- Four are cheap: `StreamQueue::enqueueValueWithSize`
(`StreamQueue.h:138`, the 67,108,864th `enqueue()`, 1.1 GB),
`BunTextAccumulator::pieces` (the 262,343,954th binary `write()` under
`.text()`, 2.1 GB), `streamingUTF8Decode` (`WebStreamsMisc.cpp:101`) and
`concatenateChunks` (`BunStreamConsumers.cpp:402`), each with one
2^31-byte input.
- Four need 8 GB or more: the read request, read-into and pull-into
`Deque`s and the byte queue (from oven-sh#42659, merged into this branch).

### Fix
- Each site throws `RangeError: Out of memory`, as oven-sh#37237 does. `Deque`
has no fallible append, so those sites check `Bun::maxDequeSize<T>()`
before the cell lock. The `Vector` sites call `tryAppend`, `tryGrow` or
`tryReserve*`.
- A failed `enqueue()` errors the stream. A refused `read()` rejects and
the stream stays readable. `writer.write()` never throws: the stream
errors and the pending writes reject. A full `tee()` branch errors both
branches and cancels the source.
- In `concatenateChunks` this PR only stops the abort. oven-sh#37239 (open)
makes totals up to 4 GiB resolve again.
- Verified: 14 new tests in `streams.test.js`, 2 in
`streams-string-limit.test.ts`. All 16 fail on the `main` sources.
Self-reviewed: 14 concerns raised, 12 addressed. The other 2 ask for a
source lint (oven-sh#42648).

### Background
- `WTF::Vector<T>` holds `INT32_MAX / sizeof(T)` elements. `append`
calls `CRASH()` past that, and `tryAppend` returns false. `WTF::Deque`
has a power-of-two capacity and keeps one slot empty.
- `Bun::maxVectorSize<T>()` and `Bun::maxDequeSize<T>()` are those
bounds, lowered by `Bun__stringSyntheticAllocationLimit`. The tests set
it to 64 KiB in a child process.
- `cellLock()` guards GC references against the concurrent marker. A
throw allocates from the GC heap, so it happens outside the lock.

<details><summary>Notes</summary>

**Repros, on a release build of `main` (f04caca) and on Bun 1.3.14**

```js
// 1. main: exit 134 after 2.6 s at 1.11 GB. 1.3.14: finishes, 4.7 GB.
let c;
new ReadableStream({ start(ctrl) { c = ctrl; } }, { highWaterMark: Infinity });
for (let i = 0; i < 67108864; i++) c.enqueue(1);

// 2. main: exit 134 after 10 s at 2.13 GB. 1.3.14: resolves, 4.4 GB.
const chunk = new Uint8Array(1);
await new ReadableStream({
  type: "direct",
  pull(c) { for (let i = 0; i < 262343954; i++) c.write(chunk); c.end(); },
}).text();

// 3. main: exit 134 at once, 29 MB. 1.3.14: exits 0.
const w = new TextDecoderStream().writable.getWriter();
w.write(new Uint8Array([0xe2]));
w.write(new Uint8Array(2 ** 31));

// 4. main: exit 134 at once, 32 MB. 1.3.14: resolves with 2,200,000,001 bytes.
const big = new Uint8Array(1100000000);
await Bun.readableStreamToArrayBuffer(new ReadableStream({
  start(c) { c.enqueue("a"); c.enqueue(big); c.enqueue(big); c.close(); },
}));
```

**Provenance**
- Sites 1 to 3 come from one fuzzing census of `main`. Site 4 is the
abort oven-sh#37239 reported on 2026-08-09. A review of this diff found that it
was still open in a file this PR edits.
- The four expensive `Deque`s come from the same census. This PR first
left them out and listed them in oven-sh#42648. oven-sh#42659 then fixed them on top
of this branch, and it was merged into it.
- All eight containers first appear in oven-sh#33193, the C++ rewrite of the
streams. Before it the queues were JS arrays, which throw `RangeError:
Out of memory`.
- This is hygiene under the `REVIEW.md` rule that a failure user input
can reach is a catchable error. It is not urgent.

**With this PR, on a release build, at the real limits** (measured
before oven-sh#42659 was merged in)
- Repro 1: `enqueue()` throws at 67,108,862 values. Exit 0, 2.9 s, 1.08
GB.
- Repro 2: `write()` throws at 262,343,953 pieces, then `text()`
resolves with that many characters. Exit 0, 12.1 s, 2.67 GB peak.
- Repro 3: the second `write()` rejects. Exit 0, 0.1 s, 20 MB.
- A `WritableStream` whose queue holds exactly 67,108,862 writes closes
and drains (5.7 GB). One more write errors the stream: `writer.closed`
and all 67,108,863 write promises reject with the `RangeError`, and no
chunk reaches the sink (3.75 GB).
- That last run gave the write promises no reactions. With a reaction on
each, `writableStreamFinishErroring` queues 67 million jobs in one loop,
and JSC's microtask queue aborts at the 33,554,432nd pending job. A
plain `Promise.resolve().then()` loop does the same on `main` and on
1.3.14. oven-sh#42648 tracks it.
- The four expensive `Deque`s were not run at their real limits (8.5 GB
to 42 GB). The synthetic limit covers them.

**Where each number comes from**
- A value queue entry is 16 bytes, so the largest legal `Deque` capacity
is 2^26. A ring of capacity C holds C - 1 entries: 67,108,863. After
this change that is 67,108,862 values plus the slot for the
`WritableStream` close sentinel. A queue takes one sentinel at most,
because `close()` rejects when a close is already queued.
- A piece is 8 bytes. `Vector::append` grows by a quarter, and the step
from 262,343,953 passes the largest capacity (268,435,455). `tryAppend`
refuses at the same point. `maxVectorSize` binds only when a test lowers
the limit.
- `concatenateChunks` also keeps a 16-byte entry per chunk. It now
reserves them in one fallible step.
- A read request, a read-into request and a pull-into descriptor each
take an 8-byte slot: 134,217,727 entries. A byte queue entry is 24
bytes: 67,108,863 entries.

**What each refused operation leaves behind**
- The byte controller checks the pull-into `Deque` before it transfers
the caller's buffer, and it adds the request before the descriptor. A
refused `read(view)` keeps its view and leaves no descriptor.
- `readableByteStreamControllerEnqueueChunkToQueue` errors the stream
when the byte queue is full. The buffer of the chunk is already
transferred at that point, so the chunk cannot be handed back. The same
function does this for a failed clone.
- A direct stream refuses the read before `pull()` runs, so a `close()`
that `pull()` defers is not lost.
- `teeAbortWithError` is the clone-failure path of the byte tee, moved
into one function. The default tee and the byte tee now use it when a
branch enqueue throws.

**Sites left as on `main`**
- The write request `Deque` (`WritableStreamOperations.cpp:264`). The
value queue of the same stream fills first, and `enqueueValueWithSize`
bounds that.
- JSC's microtask queue (oven-sh#42648).
- `StreamQueue::prepend` has no caller.

**`VectorSizeLimit.h`**
- It started as the file oven-sh#42214, oven-sh#40577 and oven-sh#42220 add, byte for byte.
oven-sh#42659 added `maxDequeSize` and `#include <bit>` to it. Whichever of
those PRs lands after this one takes this copy, which is a superset.

**The tests**
- Fourteen tests run children with the 64 KiB limit, the knob
`scrypt.test.ts`, `argon2.test.ts` and oven-sh#42214 use. On the `main` sources
every loop runs to its end and reports no error, or a refused read never
rejects. So the fail-before is an assertion diff, not an abort, and no
test runs into its timeout.
- The `TextDecoderStream` test and the `arrayBuffer()` byte-total test
need the real sizes: a later check in the same function has the same
threshold as the synthetic limit. Each child reserves its big chunk and
never touches it. On the `main` sources each child exits 134.
- The `WritableStream` test also closes a writer whose queue is exactly
full, to cover the sentinel slot.
- `streams-leak.test.ts` fails one RSS threshold on a debug ASAN build
in this container, with and without this diff (761 MB against a 700 MB
bound on the `main` sources).
</details>

<!-- robobun:evidence:begin -->

---

**no test proof** · iteration 0 · platform-specific test(s) that do not
run on this machine, deferring to CI, which covers all platforms:
test/js/web/streams/streams.test.js

<!-- robobun:evidence:end -->

---------

Co-authored-by: robobun <robobun@bun.sh>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants