Conversation
WalkthroughThis PR reworks ChangesFFI pointer decoding and ArrayBuffer size limit
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
|
Updated 8:42 PM PT - Jul 9th, 2026
❌ @robobun, your commit 75bc1fc has 3 failures in
🧪 To try this PR locally: bunx bun-pr 33353That installs a local version of the PR into your bun-33353 --bun |
f8da7ab to
127dc7b
Compare
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Thanks — both earlier comments are addressed (the byteOffset polarity fix + test, and the at-the-limit test no longer trips ASan). I'm not auto-approving only because this changes user-visible error semantics (returned Error → thrown) and adjusts the shared ArrayBuffer::MAX_SIZE constant, which is worth a maintainer glance.
Extended reasoning...
Overview
Three files: src/jsc/array_buffer.rs (drops the u32::try_from narrowing in from_bytes/from_owned_bytes; corrects MAX_SIZE from u32::MAX to 1 << 32), src/runtime/ffi/FFIObject.rs (reworks get_ptr_slice to return JsResult<(ptr, len)> and throw on validation failure instead of returning an Error object; adds a max_byte_length bound that surfaces ERR_OUT_OF_RANGE; fixes the inverted byteOffset arms), and three new tests in test/js/bun/ffi/ffi.test.js.
Both of my earlier inline comments have been addressed in commit 5169768: the byteOffset else-if polarity is corrected (with a dedicated test covering undefined/null/garbage and CString#arrayBuffer), and the subprocess test now holds backing in a named local, drops the toBuffer half of the at-the-limit check so no foreign-memory deallocator is installed, and drains stderr without asserting it empty.
Security risks
None new. bun:ffi is inherently a trust-the-caller API operating on raw addresses; this PR tightens validation (bounds byteLength before it can trip a JSC RELEASE_ASSERT) rather than loosening anything. No auth/crypto/permissions surface.
Level of scrutiny
Medium-high. The mechanical parts (dropping dead u32::try_from(..) as usize casts, threading a max through get_ptr_slice) are straightforward, but there are two cross-cutting decisions a maintainer should ratify:
- Return → throw: every validation failure in
get_ptr_slice/to_array_buffer/to_buffer/new_cstringnow throws instead of returning anErrorobject. The description calls out thereturns: "cstring"sentinel-address consequence; that's almost certainly a bug fix, but it is a user-visible semantics change on a public API. ArrayBuffer::MAX_SIZEbump (u32::MAX→1 << 32): the description audits the other consumer (node:crypto'sMAX_POSSIBLE_LENGTH = min(MAX_SIZE, i32::MAX), unchanged either way), but it's a shared constant so worth a second pair of eyes.
Other factors
Tests cover both boundaries (2**32 succeeds, 2**32+1 throws), the throw-vs-return distinction, and the byteOffset polarity fix. The at-the-limit toArrayBuffer(address, 0, 2**32) still forms a Rust slice larger than its backing allocation — technically UB per from_raw_parts_mut's contract — but that's inherent to exercising this API without a real 4 GiB mapping, only .byteLength is read, and backing.length at the end keeps the source alive; I don't think there's a better option in-suite. The PR also flags a likely merge conflict with #32260 on the same byteOffset lines. CI on the latest commit (build #68472) is still building at the time of this review.
|
Since the review flagged two things as wanting a maintainer glance, here is the evidence for both so they are cheap to check.
The constant was already documented as Returned
The only callers of these three entry points inside the repo are in
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/js/bun/ffi/ffi.test.js`:
- Around line 723-767: This FFI subprocess test is self-contained and can run
alongside the other cases. Update the existing `it("toArrayBuffer and toBuffer
reject a byteLength past the max ArrayBuffer size", ...)` test to use
`it.concurrent(...)` so it overlaps with other FFI tests without changing the
assertions or subprocess logic.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: e4ed0eab-bbf4-475c-835e-4762598ff4b7
📒 Files selected for processing (3)
src/jsc/array_buffer.rssrc/runtime/ffi/FFIObject.rstest/js/bun/ffi/ffi.test.js
CI status: diff is green, red lanes are unrelatedRebased at Neither failure touches The recurring red:
Build 70679 finished 284 / 2, the second being Build 68514 / 68472 (original base) both finished 285 / 1, the one red being Locally the three new tests pass under I have spent my one retrigger and will not push more empty commits. This is ready for review. |
07d88c1 to
17aa758
Compare
There was a problem hiding this comment.
No issues found on the current revision — my earlier feedback is addressed. Deferring to a human since this restructures the FFI pointer-decoding path, changes user-visible error semantics (returned Error → thrown), and adjusts the shared ArrayBuffer::MAX_SIZE constant.
Extended reasoning...
Overview
This PR touches three files: src/jsc/array_buffer.rs (drops u32::try_from narrowing in from_bytes/from_owned_bytes, changes MAX_SIZE from u32::MAX to 1 << 32), src/runtime/ffi/FFIObject.rs (rewrites get_ptr_slice to return JsResult<(ptr, len)> with a max_byte_length bound, converts to_invalid_arguments returns into throw_invalid_arguments, fixes the inverted byteOffset arms), and test/js/bun/ffi/ffi.test.js (three new tests covering thrown-vs-returned errors, omitted-byteOffset, and the 2**32 boundary).
Security risks
None new. bun:ffi is an inherently trusting API — the code already dereferences user-supplied raw addresses. The change tightens validation by bounding byteLength before it reaches JSC's RELEASE_ASSERT, converting an abort into a catchable RangeError. No new attack surface is introduced.
Level of scrutiny
High. This is FFI/unsafe code operating on raw pointers, and it makes several non-mechanical decisions a maintainer should sign off on:
- Error semantics change:
toArrayBuffer/toBuffer/CStringargument errors now throw instead of being returned as the result value. The author audited callers (src/js/bun/ffi.ts,ffi.d.ts) and none inspect forinstanceof Error, but this is a user-visible behavior change on a public API. - Shared constant:
ArrayBuffer::MAX_SIZEmoves fromu32::MAX(4294967295) to1 << 32(4294967296). The author verified the one other consumer (node_crypto_binding.rs'smin(MAX_SIZE, i32::MAX)) is unaffected, but a human should confirm. - Widened
from_bytes: dropping the narrowing cast affects ~15 other callers (per the PR description). The author's analysis is that they previously panicked and now reach JSC's assert instead — abort either way — but this reaches beyond the FFI module.
Other factors
I left two inline comments on an earlier revision (inverted byteOffset arms; ASan bad-free in the at-the-limit toBuffer test) — both were addressed with fixes and detailed responses, and both threads are resolved. The current bug-hunting pass found nothing. Tests are thorough, follow harness conventions, and were verified to fail on USE_SYSTEM_BUN=1. CI is green except for a fleet-wide darwin artifact-download timeout unrelated to this diff. The PR also flags a likely conflict with #32260 on the same byteOffset lines.
… huge byteLength
toArrayBuffer(ptr, 0, byteLength) aborted the process for any byteLength at
or above 2^32: ArrayBuffer::from_bytes narrowed the length through
u32::try_from(..).expect("int cast"), a leftover from when the descriptor's
len/byte_len were u32. They are usize now, and the C ABI they mirror
(Bun__ArrayBuffer) has always used size_t, so the cast only served to panic.
toBuffer aborted the same way past 2^32, there by tripping JSC's
RELEASE_ASSERT(m_sizeInBytes <= MAX_ARRAY_BUFFER_SIZE).
Drop the narrowing casts and bound the byteLength in the FFI layer instead.
The limit is MAX_ARRAY_BUFFER_SIZE (2^32 inclusive, what new ArrayBuffer(2**32)
and require("buffer").kMaxLength already accept), so a caller with a mapping
larger than that now gets a RangeError it can handle by windowing. The bound
covers the NUL-scan path too, not just an explicit byteLength.
ArrayBuffer::MAX_SIZE is documented as kMaxLength but held u32::MAX; correct it
and use it. Its one other consumer, node:crypto's MAX_POSSIBLE_LENGTH =
min(MAX_SIZE, i32::MAX), is unchanged.
CString keeps the address bound: its result is capped by WTF::String::MaxLength
in code units, which a UTF-8 byte count does not map onto.
get_ptr_slice handed its validation failures back as Error objects in the
return slot rather than throwing them, which would have left the new RangeError
uncatchable. Make them throw, along with the finalizer-argument checks in
toArrayBuffer/toBuffer. A "cstring" symbol whose C function returns one of the
sentinel addresses now throws from the call rather than yielding a CString
whose text is the error message.
The two else-if arms were inverted, so `toArrayBuffer(ptr, undefined, len)` errored while `toArrayBuffer(ptr, "garbage", len)` silently ignored the bad offset. CString#arrayBuffer passes a byteOffset that defaults to undefined, so it cached a TypeError as its ArrayBuffer. `ptr()` already has the right polarity; match it. The byteLength test held a toBuffer view over a pointer bun does not own. toBuffer without a finalizer installs MarkedArrayBuffer_deallocator, which is libc free under cfg(bun_asan), and BUN_DESTRUCT_VM_ON_EXIT=1 on the ASan lane finalizes it at teardown: a bad-free on JSC memory. toArrayBuffer installs no deallocator, so it alone holds the at-the-limit view.
17aa758 to
75bc1fc
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/runtime/ffi/FFIObject.rs`:
- Around line 648-656: Align the empty-value handling in the twin functions to
avoid inconsistent behavior. In to_array_buffer, change the ctx validation guard
to use is_empty_or_undefined_or_null(), matching to_buffer, so empty JSValue
inputs are treated as absent rather than rejected.
- Around line 536-539: Move the byteOffset finiteness validation before the
to_int64() conversion and saturating_add/sub pointer arithmetic in the relevant
FFI pointer construction logic. Reuse the existing
byte_off.as_number().is_finite() check and return the invalid-arguments error
immediately, then perform address calculation and retain the later pointer
validation.
- Around line 555-600: In the explicit length branch of the byteLength handling,
reject non-finite and fractional numeric values before calling to_int64(),
matching the validation used for byteOffset. Update the value_length checks in
the length_i computation to require a finite integer, then retain the existing
positive-value validation and cast.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: c004c153-ad99-46dc-9e5a-691e681a4cef
📒 Files selected for processing (2)
src/jsc/array_buffer.rssrc/runtime/ffi/FFIObject.rs
There was a problem hiding this comment.
Both earlier findings are addressed and I found nothing new, but the returned→thrown error semantics change and the ArrayBuffer::MAX_SIZE bump are user-visible API decisions worth a maintainer's sign-off.
Extended reasoning...
Overview
Fixes a process abort in bun:ffi's toArrayBuffer/toBuffer when byteLength exceeds JSC's MAX_ARRAY_BUFFER_SIZE. Touches three files: src/jsc/array_buffer.rs (drops u32::try_from narrowing in from_bytes/from_owned_bytes, corrects MAX_SIZE from u32::MAX to 1 << 32), src/runtime/ffi/FFIObject.rs (rewrites get_ptr_slice to return JsResult<(*mut u8, usize)> with a max_byte_length cap, converts all validation failures from returned Error objects to thrown errors, fixes the inverted byteOffset arms), and test/js/bun/ffi/ffi.test.js (three new tests).
Security risks
None introduced. bun:ffi is an inherently trusting API where the user supplies raw pointers; this change tightens validation (adds a RangeError bound where there was previously an abort) rather than loosening it. No new unsafe blocks beyond what already existed; the from_raw_parts_mut calls are unchanged in shape.
Level of scrutiny
Moderate-to-high. This is not a mechanical fix — it bundles four distinct behavior changes: (1) the headline abort→RangeError fix, (2) converting get_ptr_slice validation failures from returned-in-result-slot to thrown (affects toArrayBuffer, toBuffer, and CString — the PR description notes a returns: "cstring" symbol hitting a sentinel address now throws instead of yielding a CString whose text is the error message), (3) swapping the inverted byteOffset arms so undefined/null are accepted and non-numbers rejected, and (4) bumping the shared ArrayBuffer::MAX_SIZE constant by 1 and widening it to usize. The author has audited each (the MAX_SIZE consumer audit and the returned→thrown behavior matrix are both in the thread), and the .d.ts types already declare these functions as returning ArrayBuffer/Buffer without | Error, so (2) aligns runtime with published types. But these are still API-shape decisions on a public module that a maintainer should confirm.
Other factors
Both of my earlier inline findings (inverted byteOffset arms turning a valid CString#arrayBuffer call into a throw; the ASan-lane toBuffer finalizer freeing foreign memory at VM teardown) were fixed and the threads resolved. The from_bytes cast removal widens slightly beyond FFI (~15 other callers per the PR description), which the author characterizes as abort→abort with no new correctness hole. CI on the latest commit is reported green on all ASan shards with two unrelated infra failures. Test coverage is thorough (both boundary sides, both entry points, the byteOffset fix, and the returned-vs-thrown distinction spelled out explicitly). The PR also flags an expected merge conflict with #32260 on the same byteOffset lines.
…39564) ### Problem - More than 2^32 bytes of captured child output kill the process instead of throwing. ``await Bun.$`head -c 4294967297 /dev/zero`.quiet()`` dies with `panic(main thread): abort() called` (SIGABRT). `Bun.spawnSync({ cmd: ["head", "-c", "4294967297", "/dev/zero"] })` dies with `panic: int cast: TryFromIntError(PosOverflow)`. So does an output of exactly 2^32 bytes, which is a valid Buffer length. - `bun:ffi` `toBuffer(ptr, 0, 2 ** 32 + 1)` reaches the same abort with no memory at all. - Cause 1: `JSBuffer__bufferFromPointerAndLengthAndDeinit` (`src/jsc/bindings/JSBuffer.cpp:383` on main) and `JSBuffer__fromMmap` (`JSBuffer.cpp:2588`) hand their bytes to `ArrayBuffer::createFromBytes` without a length check. JSC RELEASE_ASSERTs there above `MAX_ARRAY_BUFFER_SIZE` (`vendor/WebKit/Source/JavaScriptCore/runtime/ArrayBuffer.cpp:150`). - Cause 2: the spawnSync output arms (`subprocess/Readable.rs:294`, `SubprocessPipeReader.rs:333`) went through `ArrayBuffer::from_owned_bytes` / `from_bytes`, which convert the length through `u32` with `expect`. That panic fires at 2^32 and above, before the bytes reach JSC. - Cause 3: once the hand-off throws, two error paths that nothing had reached before misbehave. `spawn_maybe_sync` (`js_bun_spawn_bindings.rs:2041`) returns without `finalize`, which leaks the subprocess. The shell (`interpreter.rs:1227`, added in #37275) rejects with the `JSC::Exception` cell instead of the thrown value, and the JS `reject` callback in `builtins/shell.ts` expects `(code, stdout, stderr)`, so it throws a TypeError and the promise never settles. In a debug build the child dies on an assertion in `JSCell::toStringSlowCase`. ### Fix - Both entry points in `JSBuffer.cpp` release the bytes (the deallocator, or `munmap`) and throw `RangeError: Out of memory`, the error `new ArrayBuffer(2 ** 32 + 1)` throws, for a length above `MAX_ARRAY_BUFFER_SIZE`. The `JSBuffer.cpp` and `JSBuffer.h` hunks are byte for byte the ones in #39558 (same blobs), which adds the check to the ArrayBuffer and typed array entry points at the same time. The two PRs merge in either order. The rest of this PR is what that check makes reachable. - The check belongs in the entry points because they are the last step before the assert, and every producer of a Buffer from native bytes uses them: spawnSync, `Bun.$`, `bun:ffi`, `bun:sqlite` serialize, zlib, `Bun.Archive`, `Bun.Image`. A length of exactly 2^32 still passes. JSC accepts it, and `toBuffer(ptr, 0, 2 ** 32)` already works today. - The bytes are released before the throw because the caller gave them up when it called. The `length == 0` branch of the same function already does this. The repro's RSS drops to about 340 MiB after the catch. - The two spawnSync output arms call `JSValue::create_buffer_from_box`. It makes the same C++ call as before with the same deallocator, without the `u32` hop. The `u32` conversions themselves stay as they are: #39558 removes them for the `Bun.file()` path, and this PR does not depend on that. - `spawn_maybe_sync` builds stdout, stderr and the resource usage object first, runs `finalize`, and propagates an error after that. The pre-existing return for an exception that is already pending after the wait (a termination) runs `finalize` too. `finalize` releases the other stream's buffer and the abort signal reference, and it does not run JS, so a pending exception does not affect it. - `to_js_buffer_from_memfd` and `to_js_buffer_from_fd` return `Err` when they throw. Before, they returned `Ok(JSValue::ZERO)` with the exception pending, and spawnSync would have stored that empty value in the result object. Nothing reaches this path from JS since #33832 (spawnSync no longer uses a memfd for stdout), so it has no test of its own. - The shell interpreter takes the error with `take_error`, which unwraps the `JSC::Exception`, and the JS `reject` callback forwards the value to the promise. `reject` has exactly one caller, this path. Exit codes still go through `resolve`, which builds the `ShellError`. - Verified with `test/js/bun/ffi/ffi.test.js` ("toBuffer at the Buffer length limit"): on main the child aborts, with the fix it throws a RangeError and a view of exactly 2^32 bytes is still created. This case costs no memory and runs on every platform. It only checks the error class, so it holds with #33353 too. - Verified with `test/js/bun/spawn/spawnSync.test.ts` ("spawnSync output at the Buffer length limit"): 2^32 + 1 bytes throw the RangeError, 2^32 bytes come back as a Buffer. On main both cases die with the `int cast` panic. About 11 s per case in a debug build, 8.3 GiB peak RSS in the child, so the block skips below 16 GiB of RAM and is POSIX only (`head -c`). The glibc CI test machines have 8 GB and skip it, the ASAN machines have 64 GB and run it. - Verified with `test/js/bun/shell/shelloutput.test.ts` ("stdout at the Buffer length limit"): the promise rejects with the RangeError. On main the child aborts. With the entry point check alone the child dies on the `toStringSlowCase` assertion. Same size gate. - `bun bd test` passes on `spawnSync.test.ts`, `spawn.test.ts`, `spawn-maxbuf.test.ts`, `spawnsync-isolated-event-loop.test.ts`, `spawnsync-no-microtask-drain.test.ts`, `ffi.test.js`, `bunshell.test.ts`, `shelloutput.test.ts`, `throw.test.ts` and `test/internal/source-lints`. The repros are clean under `BUN_JSC_validateExceptionChecks=1`. `cargo check` for `x86_64-pc-windows-msvc` and clippy are clean. - Related open PRs: #39558 (see above), #33353 (a check in `bun:ffi` in front of the entry point), #37243 (a check in `Buffer.from(string)` in front of it). They apply on top of this change. ### Background A Buffer is a Uint8Array over a JSC `ArrayBuffer`. JSC stores the byte length as `size_t` but caps it at `MAX_ARRAY_BUFFER_SIZE` (2^32 on 64-bit, `PageCount.h`). Bun exposes the cap as `require("buffer").kMaxLength`. JSC's allocating constructors return null above the cap, and the callers turn that into `RangeError: Out of memory`. The adopting constructor, `createFromBytes`, takes bytes that already exist and asserts instead, so the check has to happen before the hand-off. Adopting means JSC takes ownership of a byte range that native code allocated, and frees it through a deallocator passed along with the bytes when the object is collected. spawnSync and the shell use this to return their captured output without a copy. `bun:ffi` `toBuffer` uses it with a deallocator that frees nothing, which is why it can describe any length without owning that much memory. In Rust, `JsResult` is `Result<JSValue, JsError>`, and `Err(Thrown)` means an exception is pending on the VM. `from_js_host_call` turns a C++ call that returns an empty value with an exception pending into that `Err`. `take_exception` removes the pending exception and returns JSC's `Exception` cell, the wrapper that carries the thrown value and its stack. `take_error` returns the thrown value itself, which is what a JS callback has to receive. The promise helpers unwrap the cell themselves, which is why the other `take_exception` callers are fine. <details> <summary>Earlier version of this PR</summary> The first push threw `RangeError` with code `ERR_BUFFER_TOO_LARGE` from the two entry points, with its own helper. #39558 added its check to the same two functions in the meantime, with `RangeError: Out of memory` for all four entry points. This PR now carries that PR's hunks unchanged instead, and the tests expect that error. </details>
…rayBuffer limit (#39558) ### Problem - `await Bun.file(path).arrayBuffer()` on a file of exactly 2^32 bytes reads the whole file and then crashes with `panic: int cast: TryFromIntError(PosOverflow)`. The same happens through `new Response(Bun.file(path)).arrayBuffer()`. Bun 1.3.14 returns the 4294967296-byte ArrayBuffer, so this is a regression of the Rust port. - Cause: `ArrayBuffer::from_bytes` and `from_owned_bytes` (`src/jsc/array_buffer.rs:392`) convert the length through `u32` with `expect` before they store it in a `usize` field. The read path reaches them from `Blob.rs:3074` with the bytes it read. - Behind that panic sits a second failure. JSC's `ArrayBuffer::createFromBytes` RELEASE_ASSERTs when it is given more than `MAX_ARRAY_BUFFER_SIZE` (2^32) bytes. `Bun.mmap()` of a file larger than 4 GiB aborts there today, and a file of 2^32 + 1 bytes would abort there once the panic is gone. ### Fix - `from_bytes` and `from_owned_bytes` store the length as is. A file of exactly 2^32 bytes is returned whole again. - `Bun::rejectBytesNoCopyAboveArrayBufferLimit` (`JSBuffer.cpp`) rejects a length above `MAX_ARRAY_BUFFER_SIZE` with the `RangeError: Out of memory` that `new ArrayBuffer(2 ** 32 + 1)` throws. The four functions that adopt bytes from Rust call it before `createFromBytes`: `Bun__makeArrayBufferWithBytesNoCopy` and `Bun__makeTypedArrayWithBytesNoCopy` (every `ArrayBuffer::to_js*` call, and `Bun.mmap`), `JSBuffer__bufferFromPointerAndLengthAndDeinit` (`create_buffer*`, `to_node_buffer`) and `JSBuffer__fromMmap` (`to_js_buffer_from_memfd`). - The caller has already handed the bytes over, so the helper runs the deallocator before it throws. That frees the read buffer, unmaps the mapping, or drops the Blob store reference, exactly as a collection would. The typed array function already ran the deallocator when creation failed after `createFromBytes`, and the Buffer function already runs it for an empty length, so no caller releases the bytes twice. The doc comments on the Rust wrappers state this. - `ArrayBuffer__fromSharedMemfd` returns an empty value for such a length. Its only caller (`Blob.rs`, the `Clone` arm) then falls back to the copying allocation, which already throws the same RangeError. - Verified with `test/js/web/fetch/blob-oom.test.ts` ("at the 4 GiB ArrayBuffer limit"): a sparse file of 2^32 bytes comes back whole, and a file of 2^32 + 1 bytes rejects with the RangeError. On main both cases die with the panic above after reading 4 GiB. The block skips below 10 GiB of RAM and on Windows, where the file reader itself rejects files of 2^32 bytes or more with ENOMEM (`read_file.rs`, `ReadFileUV`). Each case has a 120 s timeout, like the 2 GiB cases in the same file (about 6 s each in a debug build here). The two existing 2 GiB cases get the 90 s timeout they already needed on loaded debug builds. - Verified with `test/js/bun/util/mmap.test.js`: a sparse file of 2^32 + 4096 bytes throws a RangeError from `Bun.mmap(file)` and from `{ size: 2 ** 32 + 1 }`, while `{ size: 2 ** 32 }` still maps. On main the first call aborts with SIGABRT. This case costs address space only. - The two Buffer entry points have no test of their own. The only way to reach them with such a length is more than 4 GiB of piped subprocess output (`to_js_buffer_from_memfd` is not reachable from JS today: a buffer or blob is not accepted as stdout). They call the same helper the two tests above exercise. - `bun bd test` on `blob.test.ts`, `blob-cow.test.ts`, `body.test.ts`, `zstd.test.ts`, `ffi.test.js`, `spawn.test.ts` and `buffer.test.js`: all pass. - Related open PRs. #33353 removes the same `u32` conversion for `bun:ffi` and adds a check in the FFI layer. #34119 adds a `Bun.mmap` message with the byte count in `BunObject.rs`. #37243 covers in-memory string and `InternalBlob` bodies, which take the separate `fromDefaultAllocator` path. All three still apply on top of this change. The mmap test here only checks the error name, so it holds with or without #34119. ### Background A Blob that is backed by a file is read on a thread into a Rust buffer. To return it as an `ArrayBuffer` without a copy, Bun builds an `ArrayBuffer` descriptor (`ptr`, `len`) and hands it to JSC through `Bun__makeArrayBufferWithBytesNoCopy`. JSC then owns the bytes and calls the deallocator that was passed along when the object is collected. JSC stores the size of an ArrayBuffer as a `size_t`, but caps it at `MAX_ARRAY_BUFFER_SIZE` (`PageCount.h`, 2^32 on 64-bit). Its allocating constructors return null above the cap, and Bun turns that into `RangeError: Out of memory`. The adopting constructor used for zero-copy hand-offs asserts instead, so the check has to happen in Bun before the hand-off. The `u32` conversion is the port of a `@as(u32, @intcast(len))` in the Zig version, which had the same `usize` fields. In a release build that cast was unchecked, and in practice the value passed through, which is why 1.3.14 returned the buffer. The port made it a checked conversion, so the same input now panics. <!-- robobun:evidence:begin --> --- **no test proof** · iteration 1 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/bun/util/mmap.test.js test/js/web/fetch/blob-oom.test.ts <!-- robobun:evidence:end -->
) ### Problem - `toBuffer(-1)` and `toArrayBuffer("x")` from `bun:ffi` do not throw. They return the `TypeError: ptr must be a number.` object as the call's result, so `try { toBuffer(-1) } catch {}` catches nothing. All 12 argument checks in the two functions fail this way. - `get_ptr_slice` (`src/runtime/ffi/FFIObject.rs:505`) returned the error as a `ValueOrError::Err` value. `to_buffer` and `to_array_buffer` turned it into `Ok(err)`, and their finalizer checks returned `Ok(to_invalid_arguments(..))`. ### Fix - `get_ptr_slice` returns `JsResult<(*mut u8, usize)>` and throws through `throw_invalid_arguments`. The three callers use `?`. The `CString` constructor drops its rethrow of a returned error value. - The byteOffset arms of `get_ptr_slice` were inverted: `undefined` or `null` was the error, a string was ignored. Thrown, that error would fail the valid call `toBuffer(ptr, undefined, len)`. The arms now match `ptr()`: nullish means no offset, a non-number throws. - Correct because both functions are host functions behind `wrap_host_fn!`, which maps `Err` to a pending exception. Every other `bun:ffi` entry point reports a bad argument with a thrown `TypeError`. #40732 makes the same change for `ptr()`. - A `NaN` or fractional byteLength below 1 truncated to 0 and returned an empty view. The length check now tests the truncated integer with `<= 0`. - Verified: `test/js/bun/ffi/ffi-error-messages.test.ts` (34 new cases, all fail on 1.4.1). Also `ffi.test.js`, `addr32.test.ts`, `cc.test.ts`. ### Background - `to_invalid_arguments` creates an `ERR_INVALID_ARG_TYPE` TypeError and returns it as a value. `throw_invalid_arguments` creates the same error and sets it as the pending exception. A host function that wants JS to see a throw uses the second one and returns `Err`. - `wrap_host_fn!` is the trampoline between JSC and a Rust `JsResult` body. `Err` becomes the empty `JSValue` that JSC expects from a throwing host function. <details><summary>Notes</summary> - Bun's `expect(fn).toThrow()` also accepts a function that returns an Error, so it passes on the unfixed binary. The tests use a try/catch helper instead. - The Zig version had the same two defects (returned errors, inverted arms), so this is not a regression and the test lives next to the other FFI error tests. - #33353 converts the same errors inside a larger change (a `RangeError` for a byteLength past 4 GiB) and swaps the same arms. It has conflicted with main since July. This PR carries only the throw conversion so it can land on its own. #33353 can rebase onto it. - `returns: "cstring"` symbols go through `new CString(v)` in `src/js/bun/ffi.ts`. The C++ constructor already turned the returned error into a throw, so `CString` behavior does not change. It also maps a falsy byteOffset to 0 before the call, so the arm swap does not reach it. - User-visible changes beyond the throw: `toBuffer(ptr, undefined, len)` and `toBuffer(ptr, null, len)` now return a view (before: a TypeError object). `toBuffer(ptr, "garbage", len)` now throws (before: the offset was ignored). Same for `toArrayBuffer`. - Debug build: `ffi-error-messages.test.ts` 37 pass. `ffi.test.js` 147 pass, 1 fail: "ptr argument: ArrayBuffer cells through an FTL-compiled call site" times out at 5 s in the debug+ASAN build. It uses only `dlopen` and `read.u8`, and #40732 reports the same timeout without its change. `addr32.test.ts`, `cc.test.ts` and the FFI regression tests (#21677, #25231, #29181) pass. </details>
|
#40751 (merged as fd7d527) landed two parts of this PR on their own: the conversion of the |
Closing: superseded by fixes that landed on
|
| case | this PR | main today |
|---|---|---|
toArrayBuffer(p, 0, 2 ** 33) |
RangeError [ERR_OUT_OF_RANGE] |
RangeError: Out of memory (catchable, no abort) |
toBuffer(p, 0, 2 ** 32 + 1) |
RangeError [ERR_OUT_OF_RANGE] |
RangeError: Out of memory (catchable, no abort) |
toArrayBuffer(p, 0, 2 ** 32) |
works | works |
toArrayBuffer(0) |
throws | throws |
toArrayBuffer(p, undefined, 8) |
works | works |
toArrayBuffer(p, "garbage", 8) |
throws TypeError |
throws TypeError |
The routes it took:
- Bun.file().arrayBuffer(): stop panicking at 4 GiB, throw above the ArrayBuffer limit #39558 dropped the
u32::try_from(len).expect("int cast")narrowing inArrayBuffer::from_bytes/from_owned_bytes(the panic in this PR's title) and addedrejectBytesNoCopyAboveArrayBufferLimiton the C++ side, which is where theRangeErrornow comes from. Under the ASan lane'sBUN_DESTRUCT_VM_ON_EXIT=1environment the failingtoBuffercall exits cleanly, so the free-foreign-memory footgun does not fire on that path either. - bun:ffi: throw the argument errors of toBuffer and toArrayBuffer #40751 converted
get_ptr_slice's validation failures from returnedErrorobjects to thrown ones and, by restructuring thebyteOffsetarms, fixed the inverted polarity that hadundefinedrejected and non-numbers silently accepted. - bun:ffi: use the engine-native FFI when available #35246 rewrote the JS side of
CString, so theCString#arrayBuffergetter that cached aTypeErrorno longer exists in that form.
main also carries the boundary test this PR would have added (throws a RangeError above 2^32 bytes and still creates a view of exactly 2^32 bytes in test/js/bun/ffi/ffi.test.js).
Two minor differences that I am deliberately not re-opening a PR for:
- The error is
RangeError: Out of memorywith no.code, rather thanERR_OUT_OF_RANGEnamingbyteLengthand the limit. Catchable is what mattered. If a more specific message is wanted, that belongs inrejectBytesNoCopyAboveArrayBufferLimit, which is a different change. ArrayBuffer::MAX_SIZEis stillc_uint::MAX, one below JSC'sMAX_ARRAY_BUFFER_SIZEdespite being documented askMaxLength. It has since gained a second consumer (BunObject.rsuses it as the decompressionmax_outputcap), so correcting it is no longer a no-op and deserves its own audit rather than a conflict resolution.
Thanks to both reviewers for the earlier catches (the inverted byteOffset arms and the ASan toBuffer finalizer), both of which informed the shape main ended up with.
1 participant
Repro
An uncatchable
SIGABRT.toBufferaborts the same way past2 ** 32, silently, by trippingRELEASE_ASSERT(m_sizeInBytes <= MAX_ARRAY_BUFFER_SIZE)inside JSC. Both are reachable from an entirely in-contract FFI call:mmapa region bigger than 4 GiB throughdlopen'd libc and ask for a view of it.Cause
ArrayBuffer::from_bytesnarrowed the length throughu32::try_from(bytes.len()).expect("int cast"), a leftover from when the descriptor'slen/byte_lenwereu32. They areusizenow, and the C ABI they mirror (Bun__ArrayBuffer,size_t len; size_t byte_len;) has always been 64-bit, so the cast did nothing except panic. Past that,bun:ffinever bounded the user-suppliedbyteLengthagainst the largest buffer JSC can back, so a length in(2^32, 2^56)sailed into JSC and hit the release assert.JSC's
MAX_ARRAY_BUFFER_SIZEis1ull << 32inclusive on 64-bit (PageCount.h), which is alsoBun::Buffer::kMaxLength, alsorequire("buffer").kMaxLength, and also exactly whatnew ArrayBuffer(2 ** 32)accepts today.Fix
ArrayBuffer::from_bytes/from_owned_bytes.RangeError [ERR_OUT_OF_RANGE]pastMAX_ARRAY_BUFFER_SIZE, matching what every other Buffer entry point already does. The bound covers the NUL-scan path (no explicitbyteLength) as well. A caller with a larger mapping can now catch the error and window the view.ArrayBuffer::MAX_SIZEis documented askMaxLengthbut heldu32::MAX, one below the real limit. Correct it and use it. Its one other consumer isnode:crypto'sMAX_POSSIBLE_LENGTH = min(MAX_SIZE, i32::MAX), which isi32::MAXeither way.CStringkeeps the address bound: its result is capped byWTF::String::MaxLengthin code units, which a UTF-8 byte count does not map onto, soMAX_ARRAY_BUFFER_SIZEwould be the wrong number there.get_ptr_slicehanded its validation failures back asErrorobjects in the return slot rather than throwing them, so a newRangeErrorwould have been uncatchable (toArrayBuffer(0)returned aTypeErrorobject where anArrayBufferwas expected). They throw now, as do the finalizer-argument checks intoArrayBuffer/toBuffer. One user-visible consequence: areturns: "cstring"symbol whose C function hands back one of the sentinel addresses (0xDEADBEEF,0xAAAAAAAA) now throws from the call instead of yielding aCStringwhose text is the error message.Removing the cast widens slightly beyond FFI: the ~15 other callers of
ArrayBuffer::from_bytes(Blob.arrayBuffer()on a >4 GiB file being the only realistic one) previously panicked on such a length and now reach JSC's own release assert. Abort either way, no new correctness hole, and every one of them hands the descriptor straight to ato_js*sink rather than reading the length back.Also: an omitted byteOffset was the error case
While restructuring
get_ptr_sliceit turned out its twobyteOffsetarms are inverted, whichptr()a few lines up gets right:Nullish meant "error" and a non-number meant "ignore it".
CString.prototype.arrayBufferpasses abyteOffsetthat defaults toundefined, so it has been handing back (and caching) aTypeErrorwhere anArrayBufferbelongs. Swapping the arms fixes both, and it has to happen here: converting these errors from returned to thrown would otherwise turn that into a throw on a valid call.Verification
2 ** 32keeps working,2 ** 32 + 1throws, against a real 6 GiB anonymous mapping:dlopen + mmap
Three tests in
test/js/bun/ffi/ffi.test.js, all of which fail on the released binary (USE_SYSTEM_BUN=1 bun test, the byteLength one by aborting the child) and pass onbun bd test. The byteLength test pins both ends of the boundary:2 ** 32succeeds,2 ** 32 + 1throughNumber.MAX_SAFE_INTEGERthrow for both entry points.Note for reviewers: #32260 touches the
byteOffsetlines ofget_ptr_slicefor an unrelated negative-offset panic, so expect a small conflict if both land.no test proof · iteration 8 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/bun/ffi/ffi.test.js