Throw instead of aborting when stream text() consumers accumulate more than 2^31-1 bytes - #37237
Conversation
WalkthroughChangesStream text consumers now enforce the maximum accumulated string size and use fallible allocation and append operations. Tests cover queue-backed streams, direct streams, Stream text accumulation safety
Possibly related PRs
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
There was a problem hiding this comment.
Additional findings (outside current diff — PR may have been updated during review):
-
🟡
src/jsc/bindings/webcore/streams/BunStreamConsumers.cpp:738-740— TheappendChunkBytesloop infinishTextAccumulatorreturns viaRETURN_IF_EXCEPTION/if (!appended) returnwithout callingreleaseAccumulated(), while every other error return in this function — including the three this PR just added at ~728, ~734, ~753 — does. The PR's newtryGrow/tryAppendOOM throws insideappendChunkBytesnow flow through this un-released return, so for consistency with the function's own "release at every return below" comment it'd be worth callingreleaseAccumulated()before both loop returns too. (Delayed release only, not a leak — the sole caller's owner sink is a fresh local — so not blocking.)Extended reasoning...
What the issue is
finishTextAccumulator(BunStreamConsumers.cpp) opens with the invariant comment "release at every return below (the owner can outlive this call by a lot)" and defines areleaseAccumulatedlambda that resetsaccumulator.pieces(the WriteBarrier'd chunk buffers rooted on the owner cell). The pieces loop at lines 738-746, however, has two returns that do not call it:for (auto& piece : accumulator.pieces) { JSValue value = piece.get(); if (!value) continue; bool appended = appendChunkBytes(vm, globalObject, value, bytes); RETURN_IF_EXCEPTION(scope, WTF::String()); // <- no releaseAccumulated() if (!appended) return WTF::String(); // <- no releaseAccumulated() }
Every other error return in the function calls
releaseAccumulated()first — including the three new returns this PR adds (theestimatedLength > MaxLengththrow at ~728, thetryReserveInitialCapacityfailure at ~734, and the trailing-ropetryAppendfailure at ~753).The code path this PR opens
Before this PR,
appendChunkBytescould only throw viaasString(chunk)->value(globalObject)(rope resolve OOM) or the trailingthrowTypeErrorfor a bad chunk type. This PR adds three new OOM throw sites insideappendChunkBytes:bytes.tryGrow(oldSize + byteLength)failing on a string chunk whose UTF-8 expansion pushes past the reserved capacity (line ~333)bytes.tryAppend(view->span())failing for an ArrayBufferView chunk (line ~347)bytes.tryAppend(impl->span())failing for an ArrayBuffer chunk (line ~354)
All of these set the pending exception and return
false, which flows through the loop'sRETURN_IF_EXCEPTION(scope, WTF::String())at line 743 — without releasing the accumulator. So the PR simultaneously (a) added new OOM paths that reach this return and (b) addedreleaseAccumulated()to the three sibling error returns immediately above and below it, but left the loop return itself as-is.Why the impact is minimal (nit-level)
This is a delayed release, not a leak, and in practice releases nothing sooner:
- The only caller of
finishTextAccumulatorisconvertChunksToText(line 630), where the owner is a fresh localJSBunStandaloneTextSinkcreated two lines earlier. On the exception return, control unwinds out ofconvertChunksToTextand the sink becomes unreachable — it (and itsm_accumulator.pieces) is collected at the next GC cycle. - On this error path the same chunk buffers are also still held by the caller's
MarkedArgumentBuffer valuesand by the originalchunksJSArray (releaseInternalChunkArrayonly runs on the success path ofconvertChunkArrayPromise). So even ifreleaseAccumulated()ran here, the underlying ArrayBuffers would still be rooted until the caller returns. - The loop-return code itself is pre-existing unchanged context — the diff shows these lines as context, not additions.
appendChunkBytescould already throw before this PR (rope-resolve OOM, type error), so the un-released return already existed; the PR extends its reachable surface rather than introducing it.
Step-by-step walkthrough
- A queue-backed stream enqueues one small binary chunk followed by one large string chunk (e.g. 1.5 GB of UTF-16 code units that expand to >2 GB of UTF-8), so
estimatedLengthstays underMaxLengthand the mixed-chunk arm runs. convertChunksToTextcreates a freshJSBunStandaloneTextSink, writes both chunks intosink->m_accumulator, then callsfinishTextAccumulator.estimatedLength(~1.5e9) passes both new guards;tryReserveInitialCapacity(1.5e9)succeeds.- The pieces loop calls
appendChunkByteson the string chunk;utf8ByteLengthWithReplacementreports ~3 GB;bytes.tryGrow(oldSize + 3e9)fails;throwOutOfMemoryErrorsets the pending exception;appendChunkBytesreturnsfalse. - Back in the loop,
RETURN_IF_EXCEPTION(scope, WTF::String())returns immediately — without callingreleaseAccumulated().sink->m_accumulator.piecesstill holds two WriteBarrier'd JSValues. convertChunksToTextsees the exception and returns{}; the sink local goes out of scope. At the next GC the sink and its pieces are collected. The chunk buffers were also still rooted byvaluesand the caller's chunks array anyway.
How to fix
Call
releaseAccumulated()on both loop exits, matching the sibling returns:bool appended = appendChunkBytes(vm, globalObject, value, bytes); if (scope.exception() || !appended) [[unlikely]] { releaseAccumulated(); RETURN_IF_EXCEPTION(scope, WTF::String()); return WTF::String(); }
(or equivalently expand
RETURN_IF_EXCEPTIONinline and add the release before both returns). This closes the gap with the function's stated "release at every return below" invariant and REVIEW.md's "same error-path sequence as sibling exit sites" rule.
|
Good catch on the append-loop exit: it was the one error return in finishTextAccumulator that skipped releaseAccumulated(), and the new out-of-memory throws in appendChunkBytes route through it. Fixed in ed3b2b3 so that path releases the accumulator like the sibling exits. |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/jsc/bindings/webcore/streams/BunStreamConsumers.cpp`:
- Around line 330-334: Reject UTF-8-expanded input before allocation in
BunStreamConsumers.cpp:330-334 by validating the encoded byte total before
tryGrow. In BunStreamConsumers.cpp:718-751, replace unchecked rope.utf8() with a
sized, fallible encoding path that enforces the byte limit. In
JSDirectStreamController.cpp:274-315, size each string before String::utf8() and
reject aggregates exceeding the limit before allocation.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 878d3add-b756-4f18-8cbe-13e12f9265ef
📒 Files selected for processing (3)
src/jsc/bindings/webcore/streams/BunStreamConsumers.cppsrc/jsc/bindings/webcore/streams/JSDirectStreamController.cpptest/js/web/streams/streams-string-limit.test.ts
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/jsc/bindings/webcore/streams/BunStreamConsumers.cpp`:
- Around line 345-347: Validate the current binary span size before every binary
tryAppend: in src/jsc/bindings/webcore/streams/BunStreamConsumers.cpp:345-347
and src/jsc/bindings/webcore/streams/JSDirectStreamController.cpp:292-312, check
addition overflow and exceedsStringLimit(bytes.size() + span.size()) before
vector growth, preserving the catchable out-of-memory error path. In
test/js/web/streams/streams-string-limit.test.ts:81-133, add queue-backed and
direct-stream cases that resize a small resizable ArrayBuffer beyond the limit
after enqueue/write and assert RangeError: Out of memory.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: b2f29f1f-ff11-4b87-8c36-852a9f111b51
📒 Files selected for processing (4)
src/jsc/bindings/webcore/streams/BunStreamConsumers.cppsrc/jsc/bindings/webcore/streams/JSDirectStreamController.cppsrc/jsc/bindings/webcore/streams/WebStreamsInternals.htest/js/web/streams/streams-string-limit.test.ts
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/js/web/streams/streams-string-limit.test.ts`:
- Around line 137-148: Update the stream test around the ab.resize call in
pull() to print a distinct success marker immediately after
ab.resize(2400000000) returns, then require that marker in stdout alongside the
expected consumer error. Keep the existing rejection assertion, ensuring the
test only passes when resizing succeeds and Bun.readableStreamToText(rs)
subsequently rejects through the intended direct-stream append path.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: da956ed2-c763-4baa-85e0-015135642f6a
📒 Files selected for processing (1)
test/js/web/streams/streams-string-limit.test.ts
|
Updated 7:25 PM PT - Aug 16th, 2026
✅ @robobun, your commit 0189c88f42c2cafb55d8f86609567517babca1b8 passed in 🧪 To try this PR locally: bunx bun-pr 37237That installs a local version of the PR into your bun-37237 --bun |
There was a problem hiding this comment.
I reviewed this PR (including the post-9df4ceb changes) and found no bugs. Because it touches ~100 lines of allocation-sensitive C++ in the streams text-consumer paths and adds a new cross-file encoding helper, a human look would still be worthwhile.
What was reviewed:
appendUTF8WithinStringLimit: verifiedoldSize + byteLengthcannot overflowsize_t(both operands bounded well below 2^63) and thatwriteUTF8is bounded by the grown span.finishTextAccumulator/finishTextSink: confirmed the double-compare againstStringImpl::MaxLengthguards thesize_tcast, and every new error return releases the accumulator (directly, or viaendTextSink's unconditional reset).tryAppendon binary spans: confirmed it validates capacity before allocating, so a resizable buffer grown past the limit betweenwrite()andend()rejects catchably (covered by the new test).- Tests: subprocess pattern drains all pipes concurrently, asserts the combined
{stdout, stderr, exitCode}, andskipIf-gates on 8GB RAM.
Extended reasoning...
Overview
The PR converts three uncatchable abort() paths in the stream text consumers into catchable RangeError: Out of memory rejections when accumulated chunks exceed StringImpl::MaxLength (2^31-1). It touches finishTextAccumulator and appendChunkBytes in BunStreamConsumers.cpp, finishTextSink in JSDirectStreamController.cpp, adds a shared appendUTF8WithinStringLimit helper declared in WebStreamsInternals.h, and adds a 6-case subprocess test file.
Since my prior inline comment (10:41Z, addressed in 5d81613), commits 9df4ceb–af1ee12 substantially extended the fix: the String::utf8() scratch-allocation abort was identified as a third crash path and replaced with a simdutf size-then-write helper at every string-piece and rope encode site, and three more tests were added (big-ASCII-fits, UTF-8-expansion-overflows, resize-after-write). All review-thread items — mine, comment-cop's, and CodeRabbit's three findings — are resolved.
Security risks
None identified. The change strictly tightens validation: sizes are checked against StringImpl::MaxLength before allocation, and all Vector growth switches to fallible try* variants. No new user-controlled data reaches a parser or path; the error is the same throwOutOfMemoryError these consumers already throw at the string limit.
Level of scrutiny
Medium-high. This is C++ in the JSC bindings streams subsystem — allocation sizing, WTF::Vector capacity semantics, and UTF-8 byte-length arithmetic on untrusted stream data. The transformation is mechanical (crashing ops → try* + pre-checks) and the new helper is a straightforward extraction of existing size-then-write logic, but the file's memory-safety stakes and the number of call sites changed put it past what I'd auto-approve.
Other factors
- The evidence block shows all 6 tests fail on main (ASAN and release) and pass with the fix; the sibling
streams.test.js/body.test.ts/ etc. suites still pass. - The new test file runs ~15s under ASAN and
skipIf-gates on available memory, so it won't flake on small runners. - The one behavioral edge I raised earlier (direct sink rejects when >2GiB is written then detached before
end()) was intentionally kept as a rejection with a corrected comment — the old silently-drop behavior was itself wrong. concatenateChunks(thetoArrayBuffer/toBytessibling) still has the same unboundedreserveInitialCapacity/grow; the PR description explicitly scopes that to #37239, which is reasonable since ArrayBuffer results may legitimately exceed 2^31-1.
There was a problem hiding this comment.
I reviewed this PR and the bug-hunting pass found no issues. Both of my earlier inline comments (undrained stderr in the test helper; the direct-sink estimate-overcount edge) have been addressed, as have the CodeRabbit findings on String::utf8() scratch aborts and resizable-buffer growth. Because this is non-trivial C++ touching allocation and size-limit paths in the streams text consumers, a maintainer sign-off would still be worthwhile.
Checked: appendUTF8WithinStringLimit size arithmetic (oldSize+byteLength cannot overflow size_t; exceedsStringLimit gates before tryGrow); finishTextAccumulator now releases the accumulator on every error exit including the append-loop path; finishTextSink error exits are covered by endTextSink's unconditional m_textAccumulator.reset(); the six subprocess tests drain stdout/stderr concurrently, assert the combined {stdout,stderr,exitCode} object, and the resize test now proves ab.resize() succeeded before the rejection.
Extended reasoning...
Overview
The PR converts three process-abort paths in the stream .text() consumers into catchable RangeError: Out of memory when accumulated bytes exceed StringImpl::MaxLength (2^31-1). It touches BunStreamConsumers.cpp (finishTextAccumulator, appendChunkBytes, new appendUTF8WithinStringLimit), JSDirectStreamController.cpp (finishTextSink), a header declaration in WebStreamsInternals.h, and adds test/js/web/streams/streams-string-limit.test.ts with six subprocess tests gated on ≥8GB RAM.
Security risks
None introduced. The change is defensive: it replaces WTF::Vector CRASH() and WTF::String::utf8() RELEASE_ASSERT sites with fallible tryReserveInitialCapacity/tryGrow/tryAppend and an up-front exceedsStringLimit check, so oversized user input now rejects instead of aborting. The size addition in appendUTF8WithinStringLimit (oldSize + byteLength) cannot overflow on 64-bit (both operands are bounded well below 2^33). The estimate check compares in double first before casting to size_t.
Level of scrutiny
Moderate-to-high. This is C++ bindings code in a hot user-facing subsystem, with size arithmetic on user-controlled data — exactly the class REVIEW.md flags for careful review. The change itself is narrow and strictly-more-defensive (every previous unchecked append/grow/utf8() becomes a checked-and-throw), so the correctness risk is low, but it is not a mechanical change: it required understanding WTF::Vector's INT32_MAX capacity cap, String::utf8()'s 2×/3× scratch-allocation assert, and the direct sink's write()/end() timing gap for resizable buffers. That is enough non-obvious reasoning that a maintainer should confirm the approach.
Other factors
- The PR went through several bot-review rounds (comment-cop on comment length; CodeRabbit on the
String::utf8()abort and resizable-buffer growth; my own on stderr draining and the direct-sink estimate invariant). All threads are resolved and the fixes are visible in the current diff. - The evidence block shows the mechanical gate passed on iteration 2 with fails-on-main (ASAN: 6 failed, release: 2 failed) / passes-on-PR (all pass) on both build profiles, and 10k+ related tests green locally.
- The one intentional behavior change I flagged earlier (direct sink now rejects when >2GiB was written then detached before
end(), where it previously silently returned the truncated live bytes) is documented in the reworded code comment and is the safer outcome. finishTextSinkdoes not call areleaseAccumulated()on its error returns, unlike its twin — verified this is correct becauseendTextSinkunconditionally resetsm_textAccumulatorafterfinishTextSinkreturns, before checking the exception scope.- The sibling
concatenateChunks(toArrayBuffer/toBytes with mixed chunks) still uses uncheckedreserveInitialCapacity/grow/append; the PR description explicitly scopes that to #37239, which is reasonable given the different result-size constraint.
|
CI state for the final sha (699e685): everything is green except the darwin 14 x64 test-bun job, which fails only on test/napi/napi.test.ts. That failure is marked pre-existing by the CI tooling (it fails identically on main and on the previous build of this branch before an empty retrigger commit) and has been reported for triage separately; nothing in this PR touches NAPI. The fail-before/pass-after evidence check and all lanes exercising the changed code passed. Ready for review. |
…g limit Bun.readableStreamToText, Response(stream).text(), and the direct-stream text sink abort the process (uncatchable SIGABRT) when the accumulated binary chunks total more than 2^31-1 bytes, even though every individual chunk fits. finishTextAccumulator guarded its reserveInitialCapacity estimate against uint32 max, but WTF::Vector<uint8_t> caps capacity at INT32_MAX and CRASH()es above it, so totals in [2^31, 2^32) aborted before the existing exceedsStringLimit() throw could run. The direct stream's finishTextSink had no guard at all and hit the same CRASH() through incremental append. Check the estimated length against WTF::StringImpl::MaxLength up front (the estimate only undercounts, so exceeding the limit is final) and use tryReserveInitialCapacity/tryGrow/tryAppend for the byte vector so capacity and allocation failures surface as the same catchable out-of-memory RangeError these consumers already throw at the string limit.
A regression back to the abort writes the panic banner and ASAN report
to stderr; asserting the combined { stdout, stderr, exitCode } object
surfaces that output directly in the failure diff instead of an
empty-vs-expected stdout comparison.
appendChunkBytes can now throw out-of-memory, and the loop return it flows through was the one error exit in finishTextAccumulator that skipped releaseAccumulated().
User code runs between write() and end(), so a detach can make the estimate overcount there; the rejection is still the intended result.
…ng::utf8() String::utf8() RELEASE_ASSERTs once its conversion scratch passes INT32_MAX (2x the length for 8-bit strings, 3x for 16-bit with lone surrogates), so a mixed stream with a near-limit string chunk still aborted inside the encode even when the UTF-8 total fit the string limit. Add appendUTF8WithinStringLimit: size with the simdutf sizer, reject past the string limit, then write straight into the byte vector (lone surrogates become U+FFFD, matching the chunk appenders). Use it for string pieces and the trailing rope in both finish arms.
699e685 to
0189c88
Compare
There was a problem hiding this comment.
I reviewed this PR and didn't find any bugs; the one inline nit I raised earlier (the direct-sink estimate comment) was addressed. Because this is native C++ touching size/capacity arithmetic and allocation-failure semantics in the streams text consumers, a human look would still be worthwhile.
What was reviewed:
appendUTF8WithinStringLimit: overflow ofoldSize + byteLength— bounded well below SIZE_MAX (vector capped at INT32_MAX, string UTF-8 length ≤ 3·MaxLength).estimatedLengthdouble→size_t cast — guarded by the preceding> static_cast<double>(MaxLength)check, so the cast is in range.- Accumulator release on the new error exits in
finishTextAccumulator;finishTextSink's caller already resets on all paths. tryAppend/tryGrowcover the resizable-buffer-grown-after-write case without a redundant pre-check (test added).
Extended reasoning...
Overview
The PR replaces three process-abort paths in the stream text consumers with catchable RangeError: Out of memory when accumulated bytes exceed StringImpl::MaxLength (2^31-1). It touches finishTextAccumulator in BunStreamConsumers.cpp, finishTextSink in JSDirectStreamController.cpp, adds a shared appendUTF8WithinStringLimit helper (declared in WebStreamsInternals.h), and adds six subprocess tests in test/js/web/streams/streams-string-limit.test.ts. The mechanical changes are: (1) an up-front exceedsStringLimit check on the code-unit estimate, (2) fallible tryReserveInitialCapacity/tryGrow/tryAppend in place of crashing reserveInitialCapacity/grow/append, (3) simdutf-sized direct UTF-8 writes replacing WTF::String::utf8() (whose scratch allocation RELEASE_ASSERTs past INT32_MAX), and (4) releasing the accumulator on the append-loop error exit that previously skipped it.
Security risks
None identified. This is a DoS-class fix (uncatchable abort → catchable rejection) on user-controlled stream input; it does not introduce new parsing, does not relax any bounds check, and validates sizes before allocating. The size arithmetic was checked for overflow: oldSize + byteLength in the new helper is bounded by INT32_MAX + 3·INT32_MAX on 64-bit; the static_cast<size_t>(estimatedLength) is only reached after the double comparison against MaxLength passes, so the cast cannot be out of range.
Level of scrutiny
High. This is native C++ in the Web Streams subsystem touching size/index arithmetic on external data — the category REVIEW.md flags as most-blocked. The changes look correct and the test coverage is thorough (queue-backed, direct, Response().text(), mixed ASCII under limit, mixed non-ASCII whose UTF-8 expansion exceeds the limit, and a resizable buffer grown past the limit between write() and end()), with fail-before/pass-after evidence on debug+ASAN. But this is not a simple/mechanical change and warrants a maintainer sign-off.
Other factors
The PR went through three gate iterations with substantive fixes applied in response to automated feedback: paragraph-length comments folded, the String::utf8() scratch-allocation abort was identified and fixed via the new shared helper, my earlier note on the direct-sink estimate comment was addressed by rewording, a resizable-buffer test was added, and a resize-success marker was added to that test so it can't pass for the wrong reason. All inline threads are resolved, CI is green modulo a pre-existing unrelated darwin NAPI failure, and the bug hunting system found nothing this run. The sibling concatenateChunks crash for readableStreamToArrayBuffer/readableStreamToBytes is explicitly out of scope (handled in #37239).
…ot grow (#42649) ### Problem - Eight containers in the C++ web streams grow with script input and call `CRASH()` when full: `panic(main thread): abort() called`, exit 134. Every release since #33193 aborts, and Bun 1.3.14 does not. Fuzzing found them, with no user report. - Four are cheap: `StreamQueue::enqueueValueWithSize` (`StreamQueue.h:138`, the 67,108,864th `enqueue()`, 1.1 GB), `BunTextAccumulator::pieces` (the 262,343,954th binary `write()` under `.text()`, 2.1 GB), `streamingUTF8Decode` (`WebStreamsMisc.cpp:101`) and `concatenateChunks` (`BunStreamConsumers.cpp:402`), each with one 2^31-byte input. - Four need 8 GB or more: the read request, read-into and pull-into `Deque`s and the byte queue (from #42659, merged into this branch). ### Fix - Each site throws `RangeError: Out of memory`, as #37237 does. `Deque` has no fallible append, so those sites check `Bun::maxDequeSize<T>()` before the cell lock. The `Vector` sites call `tryAppend`, `tryGrow` or `tryReserve*`. - A failed `enqueue()` errors the stream. A refused `read()` rejects and the stream stays readable. `writer.write()` never throws: the stream errors and the pending writes reject. A full `tee()` branch errors both branches and cancels the source. - In `concatenateChunks` this PR only stops the abort. #37239 (open) makes totals up to 4 GiB resolve again. - Verified: 14 new tests in `streams.test.js`, 2 in `streams-string-limit.test.ts`. All 16 fail on the `main` sources. Self-reviewed: 14 concerns raised, 12 addressed. The other 2 ask for a source lint (#42648). ### Background - `WTF::Vector<T>` holds `INT32_MAX / sizeof(T)` elements. `append` calls `CRASH()` past that, and `tryAppend` returns false. `WTF::Deque` has a power-of-two capacity and keeps one slot empty. - `Bun::maxVectorSize<T>()` and `Bun::maxDequeSize<T>()` are those bounds, lowered by `Bun__stringSyntheticAllocationLimit`. The tests set it to 64 KiB in a child process. - `cellLock()` guards GC references against the concurrent marker. A throw allocates from the GC heap, so it happens outside the lock. <details><summary>Notes</summary> **Repros, on a release build of `main` (f04caca) and on Bun 1.3.14** ```js // 1. main: exit 134 after 2.6 s at 1.11 GB. 1.3.14: finishes, 4.7 GB. let c; new ReadableStream({ start(ctrl) { c = ctrl; } }, { highWaterMark: Infinity }); for (let i = 0; i < 67108864; i++) c.enqueue(1); // 2. main: exit 134 after 10 s at 2.13 GB. 1.3.14: resolves, 4.4 GB. const chunk = new Uint8Array(1); await new ReadableStream({ type: "direct", pull(c) { for (let i = 0; i < 262343954; i++) c.write(chunk); c.end(); }, }).text(); // 3. main: exit 134 at once, 29 MB. 1.3.14: exits 0. const w = new TextDecoderStream().writable.getWriter(); w.write(new Uint8Array([0xe2])); w.write(new Uint8Array(2 ** 31)); // 4. main: exit 134 at once, 32 MB. 1.3.14: resolves with 2,200,000,001 bytes. const big = new Uint8Array(1100000000); await Bun.readableStreamToArrayBuffer(new ReadableStream({ start(c) { c.enqueue("a"); c.enqueue(big); c.enqueue(big); c.close(); }, })); ``` **Provenance** - Sites 1 to 3 come from one fuzzing census of `main`. Site 4 is the abort #37239 reported on 2026-08-09. A review of this diff found that it was still open in a file this PR edits. - The four expensive `Deque`s come from the same census. This PR first left them out and listed them in #42648. #42659 then fixed them on top of this branch, and it was merged into it. - All eight containers first appear in #33193, the C++ rewrite of the streams. Before it the queues were JS arrays, which throw `RangeError: Out of memory`. - This is hygiene under the `REVIEW.md` rule that a failure user input can reach is a catchable error. It is not urgent. **With this PR, on a release build, at the real limits** (measured before #42659 was merged in) - Repro 1: `enqueue()` throws at 67,108,862 values. Exit 0, 2.9 s, 1.08 GB. - Repro 2: `write()` throws at 262,343,953 pieces, then `text()` resolves with that many characters. Exit 0, 12.1 s, 2.67 GB peak. - Repro 3: the second `write()` rejects. Exit 0, 0.1 s, 20 MB. - A `WritableStream` whose queue holds exactly 67,108,862 writes closes and drains (5.7 GB). One more write errors the stream: `writer.closed` and all 67,108,863 write promises reject with the `RangeError`, and no chunk reaches the sink (3.75 GB). - That last run gave the write promises no reactions. With a reaction on each, `writableStreamFinishErroring` queues 67 million jobs in one loop, and JSC's microtask queue aborts at the 33,554,432nd pending job. A plain `Promise.resolve().then()` loop does the same on `main` and on 1.3.14. #42648 tracks it. - The four expensive `Deque`s were not run at their real limits (8.5 GB to 42 GB). The synthetic limit covers them. **Where each number comes from** - A value queue entry is 16 bytes, so the largest legal `Deque` capacity is 2^26. A ring of capacity C holds C - 1 entries: 67,108,863. After this change that is 67,108,862 values plus the slot for the `WritableStream` close sentinel. A queue takes one sentinel at most, because `close()` rejects when a close is already queued. - A piece is 8 bytes. `Vector::append` grows by a quarter, and the step from 262,343,953 passes the largest capacity (268,435,455). `tryAppend` refuses at the same point. `maxVectorSize` binds only when a test lowers the limit. - `concatenateChunks` also keeps a 16-byte entry per chunk. It now reserves them in one fallible step. - A read request, a read-into request and a pull-into descriptor each take an 8-byte slot: 134,217,727 entries. A byte queue entry is 24 bytes: 67,108,863 entries. **What each refused operation leaves behind** - The byte controller checks the pull-into `Deque` before it transfers the caller's buffer, and it adds the request before the descriptor. A refused `read(view)` keeps its view and leaves no descriptor. - `readableByteStreamControllerEnqueueChunkToQueue` errors the stream when the byte queue is full. The buffer of the chunk is already transferred at that point, so the chunk cannot be handed back. The same function does this for a failed clone. - A direct stream refuses the read before `pull()` runs, so a `close()` that `pull()` defers is not lost. - `teeAbortWithError` is the clone-failure path of the byte tee, moved into one function. The default tee and the byte tee now use it when a branch enqueue throws. **Sites left as on `main`** - The write request `Deque` (`WritableStreamOperations.cpp:264`). The value queue of the same stream fills first, and `enqueueValueWithSize` bounds that. - JSC's microtask queue (#42648). - `StreamQueue::prepend` has no caller. **`VectorSizeLimit.h`** - It started as the file #42214, #40577 and #42220 add, byte for byte. #42659 added `maxDequeSize` and `#include <bit>` to it. Whichever of those PRs lands after this one takes this copy, which is a superset. **The tests** - Fourteen tests run children with the 64 KiB limit, the knob `scrypt.test.ts`, `argon2.test.ts` and #42214 use. On the `main` sources every loop runs to its end and reports no error, or a refused read never rejects. So the fail-before is an assertion diff, not an abort, and no test runs into its timeout. - The `TextDecoderStream` test and the `arrayBuffer()` byte-total test need the real sizes: a later check in the same function has the same threshold as the synthetic limit. Each child reserves its big chunk and never touches it. On the `main` sources each child exits 134. - The `WritableStream` test also closes a writer whose queue is exactly full, to cover the sentinel slot. - `streams-leak.test.ts` fails one RSS threshold on a debug ASAN build in this container, with and without this diff (761 MB against a 700 MB bound on the `main` sources). </details> <!-- robobun:evidence:begin --> --- **no test proof** · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/web/streams/streams.test.js <!-- robobun:evidence:end --> --------- Co-authored-by: robobun <robobun@bun.sh> Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
…ot grow (oven-sh#42649) ### Problem - Eight containers in the C++ web streams grow with script input and call `CRASH()` when full: `panic(main thread): abort() called`, exit 134. Every release since oven-sh#33193 aborts, and Bun 1.3.14 does not. Fuzzing found them, with no user report. - Four are cheap: `StreamQueue::enqueueValueWithSize` (`StreamQueue.h:138`, the 67,108,864th `enqueue()`, 1.1 GB), `BunTextAccumulator::pieces` (the 262,343,954th binary `write()` under `.text()`, 2.1 GB), `streamingUTF8Decode` (`WebStreamsMisc.cpp:101`) and `concatenateChunks` (`BunStreamConsumers.cpp:402`), each with one 2^31-byte input. - Four need 8 GB or more: the read request, read-into and pull-into `Deque`s and the byte queue (from oven-sh#42659, merged into this branch). ### Fix - Each site throws `RangeError: Out of memory`, as oven-sh#37237 does. `Deque` has no fallible append, so those sites check `Bun::maxDequeSize<T>()` before the cell lock. The `Vector` sites call `tryAppend`, `tryGrow` or `tryReserve*`. - A failed `enqueue()` errors the stream. A refused `read()` rejects and the stream stays readable. `writer.write()` never throws: the stream errors and the pending writes reject. A full `tee()` branch errors both branches and cancels the source. - In `concatenateChunks` this PR only stops the abort. oven-sh#37239 (open) makes totals up to 4 GiB resolve again. - Verified: 14 new tests in `streams.test.js`, 2 in `streams-string-limit.test.ts`. All 16 fail on the `main` sources. Self-reviewed: 14 concerns raised, 12 addressed. The other 2 ask for a source lint (oven-sh#42648). ### Background - `WTF::Vector<T>` holds `INT32_MAX / sizeof(T)` elements. `append` calls `CRASH()` past that, and `tryAppend` returns false. `WTF::Deque` has a power-of-two capacity and keeps one slot empty. - `Bun::maxVectorSize<T>()` and `Bun::maxDequeSize<T>()` are those bounds, lowered by `Bun__stringSyntheticAllocationLimit`. The tests set it to 64 KiB in a child process. - `cellLock()` guards GC references against the concurrent marker. A throw allocates from the GC heap, so it happens outside the lock. <details><summary>Notes</summary> **Repros, on a release build of `main` (f04caca) and on Bun 1.3.14** ```js // 1. main: exit 134 after 2.6 s at 1.11 GB. 1.3.14: finishes, 4.7 GB. let c; new ReadableStream({ start(ctrl) { c = ctrl; } }, { highWaterMark: Infinity }); for (let i = 0; i < 67108864; i++) c.enqueue(1); // 2. main: exit 134 after 10 s at 2.13 GB. 1.3.14: resolves, 4.4 GB. const chunk = new Uint8Array(1); await new ReadableStream({ type: "direct", pull(c) { for (let i = 0; i < 262343954; i++) c.write(chunk); c.end(); }, }).text(); // 3. main: exit 134 at once, 29 MB. 1.3.14: exits 0. const w = new TextDecoderStream().writable.getWriter(); w.write(new Uint8Array([0xe2])); w.write(new Uint8Array(2 ** 31)); // 4. main: exit 134 at once, 32 MB. 1.3.14: resolves with 2,200,000,001 bytes. const big = new Uint8Array(1100000000); await Bun.readableStreamToArrayBuffer(new ReadableStream({ start(c) { c.enqueue("a"); c.enqueue(big); c.enqueue(big); c.close(); }, })); ``` **Provenance** - Sites 1 to 3 come from one fuzzing census of `main`. Site 4 is the abort oven-sh#37239 reported on 2026-08-09. A review of this diff found that it was still open in a file this PR edits. - The four expensive `Deque`s come from the same census. This PR first left them out and listed them in oven-sh#42648. oven-sh#42659 then fixed them on top of this branch, and it was merged into it. - All eight containers first appear in oven-sh#33193, the C++ rewrite of the streams. Before it the queues were JS arrays, which throw `RangeError: Out of memory`. - This is hygiene under the `REVIEW.md` rule that a failure user input can reach is a catchable error. It is not urgent. **With this PR, on a release build, at the real limits** (measured before oven-sh#42659 was merged in) - Repro 1: `enqueue()` throws at 67,108,862 values. Exit 0, 2.9 s, 1.08 GB. - Repro 2: `write()` throws at 262,343,953 pieces, then `text()` resolves with that many characters. Exit 0, 12.1 s, 2.67 GB peak. - Repro 3: the second `write()` rejects. Exit 0, 0.1 s, 20 MB. - A `WritableStream` whose queue holds exactly 67,108,862 writes closes and drains (5.7 GB). One more write errors the stream: `writer.closed` and all 67,108,863 write promises reject with the `RangeError`, and no chunk reaches the sink (3.75 GB). - That last run gave the write promises no reactions. With a reaction on each, `writableStreamFinishErroring` queues 67 million jobs in one loop, and JSC's microtask queue aborts at the 33,554,432nd pending job. A plain `Promise.resolve().then()` loop does the same on `main` and on 1.3.14. oven-sh#42648 tracks it. - The four expensive `Deque`s were not run at their real limits (8.5 GB to 42 GB). The synthetic limit covers them. **Where each number comes from** - A value queue entry is 16 bytes, so the largest legal `Deque` capacity is 2^26. A ring of capacity C holds C - 1 entries: 67,108,863. After this change that is 67,108,862 values plus the slot for the `WritableStream` close sentinel. A queue takes one sentinel at most, because `close()` rejects when a close is already queued. - A piece is 8 bytes. `Vector::append` grows by a quarter, and the step from 262,343,953 passes the largest capacity (268,435,455). `tryAppend` refuses at the same point. `maxVectorSize` binds only when a test lowers the limit. - `concatenateChunks` also keeps a 16-byte entry per chunk. It now reserves them in one fallible step. - A read request, a read-into request and a pull-into descriptor each take an 8-byte slot: 134,217,727 entries. A byte queue entry is 24 bytes: 67,108,863 entries. **What each refused operation leaves behind** - The byte controller checks the pull-into `Deque` before it transfers the caller's buffer, and it adds the request before the descriptor. A refused `read(view)` keeps its view and leaves no descriptor. - `readableByteStreamControllerEnqueueChunkToQueue` errors the stream when the byte queue is full. The buffer of the chunk is already transferred at that point, so the chunk cannot be handed back. The same function does this for a failed clone. - A direct stream refuses the read before `pull()` runs, so a `close()` that `pull()` defers is not lost. - `teeAbortWithError` is the clone-failure path of the byte tee, moved into one function. The default tee and the byte tee now use it when a branch enqueue throws. **Sites left as on `main`** - The write request `Deque` (`WritableStreamOperations.cpp:264`). The value queue of the same stream fills first, and `enqueueValueWithSize` bounds that. - JSC's microtask queue (oven-sh#42648). - `StreamQueue::prepend` has no caller. **`VectorSizeLimit.h`** - It started as the file oven-sh#42214, oven-sh#40577 and oven-sh#42220 add, byte for byte. oven-sh#42659 added `maxDequeSize` and `#include <bit>` to it. Whichever of those PRs lands after this one takes this copy, which is a superset. **The tests** - Fourteen tests run children with the 64 KiB limit, the knob `scrypt.test.ts`, `argon2.test.ts` and oven-sh#42214 use. On the `main` sources every loop runs to its end and reports no error, or a refused read never rejects. So the fail-before is an assertion diff, not an abort, and no test runs into its timeout. - The `TextDecoderStream` test and the `arrayBuffer()` byte-total test need the real sizes: a later check in the same function has the same threshold as the synthetic limit. Each child reserves its big chunk and never touches it. On the `main` sources each child exits 134. - The `WritableStream` test also closes a writer whose queue is exactly full, to cover the sentinel slot. - `streams-leak.test.ts` fails one RSS threshold on a debug ASAN build in this container, with and without this diff (761 MB against a 700 MB bound on the `main` sources). </details> <!-- robobun:evidence:begin --> --- **no test proof** · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/web/streams/streams.test.js <!-- robobun:evidence:end --> --------- Co-authored-by: robobun <robobun@bun.sh> Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
…e row list cannot grow (#42833) ### Problem - `Bun.wrapAnsi("a ".repeat(2 ** 26), 1)` aborts: `panic(main thread): abort() called`, exit 134. Top frames: `WTF::VectorBufferBase::allocateBuffer<FailureAction::Crash>` (Vector.h:228) <- `Vector::appendSlowCase<Bun::Row<unsigned char>>`. The output (2^27 characters) is below the string length limit. - `processLine` keeps one 32-byte `Row` for each wrapped row in a `WTF::Vector` (`src/jsc/bindings/wrapAnsi.cpp:676`). `Vector::append` calls `CRASH()` when it cannot grow: at the 51,821,029th row of one input line. - Two more Vectors in the file abort the same way: the text of one row, and the `tail` copy in `trimRowTrailingSpaces`. ### Fix - Every Vector append in the file is fallible: `tryAppend` behind `Bun::maxVectorSize<T>()`, as #42649 does. A failure returns `false` up to the binding, which throws `RangeError: Out of memory`. - `trimRowTrailingSpaces` trims in place, as `trimLeadingSpaces` does. It allocates nothing. - Outputs that fit do not change: 1.9 million seeded random inputs match. - Verified: `test/js/bun/util/wrapAnsi.test.ts`. Its 2 new tests hold 15 inputs that throw (10 of the 11 statements that grow a Vector) and 5 that return exactly at a bound. Both fail without the fix. Also the `wrapAnsi.npm`, `sliceAnsi`, `stringWidth`, `stripANSI` suites. ### Background - `WTF::Vector<T>` holds `INT32_MAX / sizeof(T)` elements and grows by half. `append` calls `CRASH()` when the next capacity passes that bound. `tryAppend` returns false. - `Bun::maxVectorSize<T>()` (`VectorSizeLimit.h`, from #42649) is that bound, lowered by `Bun__stringSyntheticAllocationLimit`. The tests set it to 64 KiB in a child, so the 2049th row reaches it. - #42225 (open) makes the output `StringBuilder` of the same function throw. It does not touch these Vectors. <details><summary>Notes</summary> **Repros, release builds on Linux x64: Bun 1.4.2 and canary 782c402 against this branch** ```js // 1. The row list. Before: exit 134 after 4.2 s, 3.2 GB peak. After: RangeError in 4.3 s, 3.0 GB. Bun.wrapAnsi("a ".repeat(2 ** 26), 1); // 51,821,028 words return 103,642,055 characters before and after. One more word is the abort. // 2. One UTF-16 row. Before: exit 134, 5.4 GB. After: RangeError, 5.1 GB. Bun.wrapAnsi("\u2603".repeat(9e8) + " b", 2 ** 31); // 3. One Latin-1 row. Before: exit 134, 5.4 GB. After: RangeError, 5.1 GB. Bun.wrapAnsi("a".repeat(1.8e9) + " b", 2 ** 31); // 4. The tail copy of the trailing trim. Before: exit 134, 9.7 GB. After: returns 900,000,001 characters, 6.7 GB. Bun.wrapAnsi("a " + "\u200b".repeat(9e8), 80); ``` **Where each number comes from** - `sizeof(Row<Char>)` is 32, so the largest legal capacity of the row list is 67,108,863. `FastMalloc::nextCapacity` grows a full Vector from 51,821,028 to 77,731,542, which passes it. `tryAppend` refuses at the same step, so the function now throws at the 51,821,029th row. `maxVectorSize` binds only when a test lowers the limit. #42649 has the same property. - Repros 2 and 3: the first word fills the row with one exact-size allocation. The separator space of the next word asks for 1.5 times that, which passes `INT32_MAX` bytes. - Repro 4: the row itself fits. `trimRowTrailingSpaces` copied the zero-width tail into a second Vector one character at a time, and that Vector failed to grow past 885,410,839 characters. **A failed call** - The rows of the current line are dropped and the function throws. Nothing is cached between calls, so the next call starts clean. - The message is the one #42649 and #37237 use. A `RangeError` with the same text comes from JSC when `"a".repeat()` runs out of memory. **Not changed here** - A UTF-16 input of more than about 976 million characters still aborts, in `WTF::StringBuilder` under `joinRowsWithAnsiPreservation`. `wrapAnsiImpl` reserves 1.1 times the input length while the builder is still 8-bit, and the first 16-bit append doubles that reserve past the string limit. It is the output builder, which #42225 owns, so it is tracked apart from this PR. - `Bun.wrapAnsi` has had these Vectors since #26061 added it. This was never correct, so the test is in the module's test file. **The tests** - Each child runs with `BUN_FEATURE_FLAG_SYNTHETIC_MEMORY_LIMIT=65536`, the knob the #42649 tests use. The bounds become 2048 rows, and 65,536 Latin-1 or 32,768 UTF-16 characters in a row. - Five inputs sit exactly at a bound and return: 2048 rows (Latin-1 and UTF-16), 65,536 Latin-1 characters, 32,768 UTF-16 characters, and a separator space as character 65,536. The same input one element longer throws. - Without the fix every input returns its normal length, so the failure is an assertion diff and not an abort. - A debugger breakpoint on every `return false` confirmed which statement fails in each case. Every one is covered except the first row of a line. That one fails only when the limit is below the 32 bytes of one `Row`. **No change for outputs that fit** - A seeded generator builds inputs from escapes (SGR, OSC 8 hyperlinks, C1, unterminated and malformed sequences), tabs, line breaks, wide, zero-width, combining and surrogate characters, with `columns` 1 to 10 and every option set. A hash of 1.9 million outputs is equal on the canary (09bb546 and 782c402) and on the debug ASAN build of this branch. - Wrapping 20 MB of text, best of 3, release builds: 225 ms before and 205 ms after at 80 columns, 302 ms and 258 ms for UTF-16 text, 189 ms and 194 ms at 1 column with `hard`. </details> <!-- robobun:evidence:begin --> --- **[human-review]** gate passed · iteration 0 · 2 files touched <details><summary>fails on main (without fix)</summary> ```console ASAN without fix: 2 FAILED $ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" test/js/bun/util/wrapAnsi.test.ts bun test v1.4.3 (09bb546) test/js/bun/util/wrapAnsi.test.ts: (pass) Bun.wrapAnsi > basic wrapping > wraps text at word boundaries [3.23ms] (pass) Bun.wrapAnsi > basic wrapping > handles empty string [2.39ms] (pass) Bun.wrapAnsi > basic wrapping > no wrapping needed [1.87ms] (pass) Bun.wrapAnsi > basic wrapping > wraps multiple words [2.51ms] (pass) Bun.wrapAnsi > basic wrapping > handles single long word [2.45ms] (pass) Bun.wrapAnsi > basic wrapping > handles columns = 0 [1.82ms] (pass) Bun.wrapAnsi > hard wrap option > breaks long words in middle [2.98ms] (pass) Bun.wrapAnsi > hard wrap option > breaks very long word [3.53ms] (pass) Bun.wrapAnsi > wordWrap option > wordWrap false disables wrapping [2.53ms] (pass) Bun.wrapAnsi > wordWrap option > wordWrap: undefined keeps word wrapping on [2.86ms] (pass) Bun.wrapAnsi > wordWrap option > wordWrap: null keeps word wrapping on [0.62ms] (pass) Bun.wrapAnsi > wordWrap option > wordWrap: 0 keeps word wrapping on [0.45ms] (pass) Bun.wrapAnsi > wordWrap opti ... (truncated) release without fix: all passed bun test v1.4.3-canary.1 (7606b00) test/js/bun/util/wrapAnsi.test.ts: (pass) Bun.wrapAnsi > basic wrapping > wraps text at word boundaries [0.05ms] (pass) Bun.wrapAnsi > basic wrapping > handles empty string [0.02ms] (pass) Bun.wrapAnsi > basic wrapping > no wrapping needed [0.01ms] (pass) Bun.wrapAnsi > basic wrapping > wraps multiple words [0.01ms] (pass) Bun.wrapAnsi > basic wrapping > handles single long word [0.01ms] (pass) Bun.wrapAnsi > basic wrapping > handles columns = 0 [0.01ms] (pass) Bun.wrapAnsi > hard wrap option > breaks long words in middle [0.02ms] (pass) Bun.wrapAnsi > hard wrap option > breaks very long word [0.04ms] (pass) Bun.wrapAnsi > wordWrap option > wordWrap false disables wrapping [0.04ms] (pass) Bun.wrapAnsi > wordWrap option > wordWrap: undefined keeps word wrapping on [0.02ms] (pass) Bun.wrapAnsi > wordWrap option > wordWrap: null keeps word wrapping on [0.01ms] (pass) Bun.wrapAnsi > wordWrap option > wordWrap: 0 keeps word wrapping on (pass) Bun.wrapAnsi > wordWrap option > wordWrap: "" keeps word wrapping on (pass) Bun.wrapAnsi > wordWrap option > wordWrap: false breaks words character-by-character [0.02ms] (pass) Bun.wrapAnsi > tr ... (truncated) ``` </details> <details><summary>passes on PR (with fix)</summary> ```console ASAN with fix: all passed $ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" test/js/bun/util/wrapAnsi.test.ts bun test v1.4.3 (09bb546) test/js/bun/util/wrapAnsi.test.ts: (pass) Bun.wrapAnsi > basic wrapping > wraps text at word boundaries [2.58ms] (pass) Bun.wrapAnsi > basic wrapping > handles empty string [1.66ms] (pass) Bun.wrapAnsi > basic wrapping > no wrapping needed [1.31ms] (pass) Bun.wrapAnsi > basic wrapping > wraps multiple words [1.64ms] (pass) Bun.wrapAnsi > basic wrapping > handles single long word [1.59ms] (pass) Bun.wrapAnsi > basic wrapping > handles columns = 0 [1.24ms] (pass) Bun.wrapAnsi > hard wrap option > breaks long words in middle [2.03ms] (pass) Bun.wrapAnsi > hard wrap option > breaks very long word [1.91ms] (pass) Bun.wrapAnsi > wordWrap option > wordWrap false disables wrapping [1.66ms] (pass) Bun.wrapAnsi > wordWrap option > wordWrap: undefined keeps word wrapping on [1.79ms] (pass) Bun.wrapAnsi > wordWrap option > wordWrap: null keeps word wrapping on [0.47ms] (pass) Bun.wrapAnsi > wordWrap option > wordWrap: 0 keeps word wrapping on [0.30ms] (pass) Bun.wrapAnsi > wordWrap opti ... (truncated) release with fix: all passed $ bun scripts/build.ts --profile=release [configured] bun-profile → bun (stripped) in 907ms (unchanged) ninja: Entering directory `/workspace/bun/build/release' [1/133] gen ErrorCode+*.h [2/133] esbuild bun-error ../../build/release/codegen/bun-error/index.js 34.9kb ../../build/release/codegen/bun-error/bun-error.css 12.8kb ⚡ Done in 23ms [3/133] gen compressed/codegen/bun-error/bun-error.css.zst [4/133] gen compressed/codegen/bun-error/index.js.zst [5/133] gen NodeModuleModule.lut.h Generating /workspace/bun/build/release/codegen/NodeModuleModule.lut.h from /workspace/bun/src/jsc/modules/NodeModuleModule.cpp [6/133] gen generated_host_exports.rs generated_host_exports.rs: 122 exports (host=5, lazy=10, generic=107, rust=0); 243 extern-C blocks audited [7/133] gen ZigGeneratedClasses.{cpp,h,rs} Found 2 classes from /workspace/bun/src/jsc/resolve_message.classes.ts - ResolveMessage (15 fields) - BuildMessage (10 fields) Found 1 classes from /workspace/bun/src/runtime/api/Archive.classes.ts - Archive (4 fields, 1 class fields) Found 2 classes from /workspace/bun/src/runtime/api/BunObject.classes.ts - ResourceUsage (8 fields) - Subprocess (20 ... (truncated) ``` </details> <details><summary>diff hotspot</summary> ``` src/jsc/bindings/wrapAnsi.cpp | 115 +++++++++++++++++++++++++------------- test/js/bun/util/wrapAnsi.test.ts | 106 +++++++++++++++++++++++++++++++++++ 2 files changed, 181 insertions(+), 40 deletions(-) ``` </details> **gate history** · 2 passed · 0 rejected · iteration 0 <details><summary>evidence per changed file</summary> ``` file reads edits tests src/jsc/bindings/wrapAnsi.cpp 4 10 15 test/js/bun/util/wrapAnsi.test.ts 7 4 15 ``` </details> <!-- robobun:evidence:end -->
Problem
Bun.readableStreamToText(rs)andnew Response(rs).text()abort the process (uncatchable SIGABRT,panic(main thread): abort() called) when the stream's binary chunks sum to more than 2^31-1 bytes, even though each individual chunk is well under the limit. The same abort happens for the direct-stream text sink (type: "direct").Node's equivalent (
new Response(rs).text()) throws a catchable error.Cause
Three distinct abort paths in the text consumers' finish arms:
finishTextAccumulator(BunStreamConsumers.cpp) guarded itsreserveInitialCapacityestimate withestimatedLength < numeric_limits<uint32_t>::max(), a check in the wrong width:WTF::Vector<uint8_t>caps capacity at INT32_MAX (isValidCapacityForVector) andCRASH()es above it inWTF::VectorBufferBase<uint8_t>::allocateBuffer, so totals in [2^31, 2^32) aborted in the reserve before the function'sexceedsStringLimit()throw was reached.finishTextSink(JSDirectStreamController.cpp) had no size guard at all and hit the sameCRASH()while growing the vector through incrementalappend.WTF::String::utf8(), whichRELEASE_ASSERTs once its conversion scratch passes INT32_MAX (2x the length for 8-bit strings, 3x for 16-bit strings containing lone surrogates). A mixed stream with a near-limit string chunk aborted in the encode even when the actual UTF-8 total fit the limit.Fix
In both finish arms:
estimatedLengthagainstWTF::StringImpl::MaxLength(via the existingexceedsStringLimit) before touching the vector. Sizes are recorded at write time (binary chunk sizes exact, string chunks as UTF-16 code units, which UTF-8 re-encoding never shrinks below), so an estimate past the limit is final and throwing early is correct. This also fails fast, before copying gigabytes.tryReserveInitialCapacity/tryGrow/tryAppendfor every vector operation so capacity and allocation failures surface as catchable errors instead ofCRASH().appendUTF8WithinStringLimit: size with the simdutf sizer, reject past the string limit before allocating, then write straight into the byte vector. This removes theString::utf8()scratch-allocation aborts, and lone surrogates now consistently become U+FFFD on every mixed-path arm (string chunks already did).finishTextAccumulatorthat skipped it.The thrown error is the same
RangeError: Out of memorythese consumers already throw at the string limit (the synthetic-limit tests intest/js/web/streams/streams.test.jspin that contract).Verification
test/js/web/streams/streams-string-limit.test.ts, five subprocess tests (the file skips on machines with less than 8GB of RAM):Response(stream).text()with 3 binary chunks summing to 2^31+1 bytes: abort before,threw RangeError Out of memorywith exit 0 afterString::utf8()encode before even though the UTF-8 total fits, now resolves with the full 1200000001-char textAll verified to fail on the unfixed build (release 1.4.0 canary and debug+ASAN with the src changes stashed) and pass with the fix. Also green locally:
streams.test.js,body.test.ts,body-stream.test.ts,direct-readable-stream.test.tsx,utf8-bom.test.ts,blob-oom.test.ts,stream-fast-path.test.ts(10k+ tests).The sibling crash for
readableStreamToArrayBuffer/readableStreamToByteswith mixed chunks (concatenateChunks, same file) is a separate consumer with different constraints (an ArrayBuffer result may legitimately exceed 2^31-1) and is handled in #37239.[review] gate passed · iteration 5 · 4 files touched
fails on main (without fix)
passes on PR (with fix)
diff hotspot
gate history · 6 passed · 0 rejected · iteration 5
evidence per changed file