Skip to content

node:buffer: match Node's input handling in isUtf8() and isAscii() - #39028

Open
robobun wants to merge 3 commits into
mainfrom
farm/59931474/buffer-isutf8-isascii-input
Open

robobun wants to merge 3 commits into
mainfrom
farm/59931474/buffer-isutf8-isascii-input

Conversation

@robobun

@robobun robobun commented Aug 15, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • isUtf8(new DataView(ab)) and isAscii(new DataView(ab)) from node:buffer scan the bytes and return a boolean. Node throws TypeError [ERR_INVALID_ARG_TYPE]: The "input" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received an instance of DataView.
  • Cause: both functions start with dynamicDowncast<JSArrayBufferView> (src/jsc/modules/NodeBufferModule.h:25 and :79 on main). In JSC a DataView is a JSArrayBufferView, so it takes the "view" branch.
  • Same functions, same contract, also off: every other rejected input gets a hand-written First argument must be an ArrayBufferView instead of Node's message above; a TypedArray whose buffer was detached throws (a code-less TypeError from isUtf8, ERR_INVALID_STATE from isAscii) where Node returns true; a detached ArrayBuffer throws with the message Invalid state: Cannot validate on a detached buffer where Node's is Cannot validate on a detached buffer.
const { isUtf8, isAscii } = require("node:buffer");
const ab = new Uint8Array([0x61, 0x62, 0x63, 0x64]).buffer;
isUtf8(new DataView(ab));  // bun 1.4.0: true    node v26.3.0: throws ERR_INVALID_ARG_TYPE
isAscii(new DataView(ab)); // bun 1.4.0: true    node v26.3.0: throws ERR_INVALID_ARG_TYPE
isUtf8({});                // bun 1.4.0: "First argument must be an ArrayBufferView"
                           // node:      'The "input" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received an instance of Object'

Fix

  • Replace the two copies of the argument handling with one helper, validateBytesOf, that each function calls with its simdutf validator. The helper is a port of Node's two layers: the JS check isTypedArray(input) || isAnyArrayBuffer(input) (lib/buffer.js#L1415-L1429) and the binding's detached check (src/node_buffer.cc#L1305-L1333).
  • The fixing condition is view && isTypedArrayType(view->type()): isTypedArrayType is true for the twelve TypedArray kinds and false for DataViewType, which is exactly Node's isTypedArray. It is the predicate BunObject.cpp and napi.cpp already use for this distinction, and the one node:buffer: reject DataView in Buffer.copyBytesFrom #35239 uses for the same bug in Buffer.copyBytesFrom.
  • Rejections go through Bun::ERR::INVALID_ARG_INSTANCE("input", "ArrayBuffer, Buffer, or TypedArray", value), which keeps the ERR_INVALID_ARG_TYPE code and renders Node's wording including the Received ... suffix. This is the helper the rest of JSBuffer.cpp uses for class-instance expectations.
  • Detached handling now sits where Node has it. Node's binding only records detachment for the ArrayBuffer form (util-inl.h#L599); a view over a detached buffer reports byteLength 0 and validates as empty, so the helper reads view->span() (JSC zeroes the length when a view is detached) and only checks impl()->isDetached() on the ArrayBuffer branch. The ArrayBuffer error is thrown with Bun::throwError(ERR_INVALID_STATE, ...) rather than Bun::ERR::INVALID_STATE, because the latter prepends Invalid state: (Node's JS-side format) and this error comes from C++ in Node; NodeSqlite.cpp makes the same distinction. The impl() null check the old isUtf8 copy had is gone: JSArrayBuffer::finishCreation dereferences impl(), so it is never null for a live wrapper, and Buffer.byteLength in JSBuffer.cpp already relies on that.
  • Node main has since gone one step further: buffer: handle detached ArrayBuffer views nodejs/node#64504 (landed 2026-08-05, not in any release yet) removes the detached ArrayBuffer throw too, so there a detached ArrayBuffer and a view over one are both treated as empty. This PR follows v26.3.0, which is what the ported upstream tests in test/js/node/test/parallel/test-buffer-isutf8.js and test-buffer-isascii.js assert (ERR_INVALID_STATE for the ArrayBuffer form). When those files are re-ported from a release that contains #64504, the isDetached() branch of the helper and the one ERR_INVALID_STATE assertion added here are the two things to drop; the detached-view behavior adopted here is the one #64504 codifies.
  • utf-8-validate's default export is this same isUtf8 host function (UTF8ValidateModule.h), so it picks up the same argument handling. Its documented argument is a Buffer, and ws and crossws only ever pass one; the ArrayBuffer acceptance and ERR_INVALID_ARG_TYPE rejections it had before were already node:buffer semantics rather than the package's.
  • Verified with test/js/node/buffer.test.js (buffer.isUtf8() input handling / buffer.isAscii() input handling): 22 of the 24 new cases fail on bun 1.4.0 (three DataView shapes, six rejected inputs with the exact message, the detached ArrayBuffer message, and the detached view) and all pass with this build; the two that pass on both are the accepted-kinds controls covering every TypedArray kind, Buffer, ArrayBuffer, SharedArrayBuffer, a view over a SharedArrayBuffer, and a sub-range view, with bytes that make isUtf8 and isAscii answer differently.
  • bun bd test test/js/node/buffer.test.js: 642 pass, 1 skip, 0 fail.
  • bun bd test/js/node/test/parallel/test-buffer-isutf8.js and test-buffer-isascii.js (Node's own tests, already ported): exit 0.
  • bun bd test test/js/first_party/utf-8-validate/utf-8-validate.test.ts: pass.
  • A 49-input probe run through both functions (every TypedArray kind, sub-range views, resizable and growable buffers, out-of-bounds views, detached buffer and view, three DataView shapes, and 20 non-buffer values) prints byte-identical output on this build and on node v26.3.0; 50 of its 98 lines differ on bun 1.4.0. Output below.

Background

  • JSArrayBufferView is JSC's base class for everything that views an ArrayBuffer: the typed arrays and DataView. dynamicDowncast<JSArrayBufferView> therefore accepts a DataView; JSC::isTypedArrayType(JSType) is the predicate that excludes it (isTypedArrayTypeIncludingDataView is the other variant).
  • JSArrayBufferView::span() is the view's bytes as {vector(), byteLength()}. When a buffer is detached JSC sets every view's length to 0 and clears its vector, and a view that fell out of bounds of a shrunk resizable buffer also reports length 0, so the span is empty in both cases and no memory is read.
  • JSArrayBuffer wraps both ArrayBuffer and SharedArrayBuffer (they differ only in impl()->isShared()), which is why one downcast covers Node's isAnyArrayBuffer. A SharedArrayBuffer cannot be detached.
  • Node's ERR_INVALID_ARG_TYPE picks its wording from the expected names: capitalized class names render as must be an instance of A, B, or C. Bun::ERR::INVALID_ARG_INSTANCE is the helper that produces that form; Bun::ERR::INVALID_ARG_TYPE with a plain string renders must be of type ....
  • Node has two flavors of ERR_INVALID_STATE: the JS one formats as Invalid state: %s, the C++ THROW_ERR_INVALID_STATE uses the message verbatim. Bun::ERR::INVALID_STATE implements the JS flavor; Bun::throwError with the code gives the C++ one.
Probe output (identical on this build and node v26.3.0)
isUtf8(Uint8Array): returns true
isUtf8(Uint8Array ascii slice): returns true
isUtf8(Uint8Array subclass): returns true
isUtf8(Buffer): returns true
isUtf8(Int8Array): returns true
isUtf8(Uint8ClampedArray): returns true
isUtf8(Int16Array): returns true
isUtf8(Uint16Array): returns true
isUtf8(Int32Array): returns true
isUtf8(Uint32Array): returns true
isUtf8(Float16Array): returns true
isUtf8(Float32Array): returns true
isUtf8(Float64Array): returns true
isUtf8(BigInt64Array): returns true
isUtf8(BigUint64Array): returns true
isUtf8(ArrayBuffer): returns true
isUtf8(empty ArrayBuffer): returns true
isUtf8(empty Uint8Array): returns true
isUtf8(resizable ArrayBuffer): returns true
isUtf8(view over shrunk resizable buffer (out of bounds)): returns true
isUtf8(length-tracking view after shrink): returns true
isUtf8(SharedArrayBuffer): returns true
isUtf8(Uint8Array over SharedArrayBuffer): returns true
isUtf8(growable SharedArrayBuffer): returns true
isUtf8(detached ArrayBuffer): throws Error ERR_INVALID_STATE "Cannot validate on a detached buffer"
isUtf8(view over detached buffer): returns true
isUtf8(DataView): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received an instance of DataView"
isUtf8(DataView sub-range): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received an instance of DataView"
isUtf8(empty DataView): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received an instance of DataView"
isUtf8(undefined): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received undefined"
isUtf8(null): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received null"
isUtf8(string): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received type string ('abc')"
isUtf8(long string): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received type string ('xxxxxxxxxxxxxxxxxxxxxxxxx...')"
isUtf8(number): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received type number (42)"
isUtf8(NaN): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received type number (NaN)"
isUtf8(bigint): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received type bigint (1n)"
isUtf8(boolean): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received type boolean (true)"
isUtf8(symbol): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received type symbol (Symbol(s))"
isUtf8(named function): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received function named"
isUtf8(anonymous arrow): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received function "
isUtf8(class instance): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received an instance of Foo"
isUtf8(plain object): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received an instance of Object"
isUtf8(object with props): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received an instance of Object"
isUtf8(null-prototype object): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received [Object: null prototype] {}"
isUtf8(array): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received an instance of Array"
isUtf8(Blob): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received an instance of Blob"
isUtf8(Date): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received an instance of Date"
isUtf8(Promise): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received an instance of Promise"
isUtf8(no argument): throws TypeError ERR_INVALID_ARG_TYPE "The \"input\" argument must be an instance of ArrayBuffer, Buffer, or TypedArray. Received undefined"

The isAscii half is the same except that the eight-byte inputs (which end in the UTF-8 encoding of é) return false; the ASCII slice, the out-of-bounds and length-tracking views, the empty inputs and the detached view return true.


no test proof · iteration 1 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/node/buffer.test.js

Both functions downcast to JSArrayBufferView, which a DataView also is,
so a DataView was scanned instead of rejected, and the rejection used a
hand-written message instead of Node's ERR_INVALID_ARG_TYPE text. The
two functions also disagreed on a view over a detached buffer (isUtf8
threw a plain TypeError, isAscii ERR_INVALID_STATE) where Node returns
true, and the detached ArrayBuffer message carried an "Invalid state: "
prefix Node's C++-thrown error does not have.

Route both through one helper that accepts a TypedArray (isTypedArrayType
excludes DataView), an ArrayBuffer or a SharedArrayBuffer, throws
ERR_INVALID_STATE with Node's message for a detached ArrayBuffer, and
ERR_INVALID_ARG_TYPE through INVALID_ARG_INSTANCE for everything else.
@coderabbitai

coderabbitai Bot commented Aug 15, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 4 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 846308b5-ad94-4808-a1eb-92c713cb1016

📥 Commits

Reviewing files that changed from the base of the PR and between 88a6398 and 2f7de0c.

📒 Files selected for processing (2)
  • src/jsc/modules/NodeBufferModule.h
  • test/js/node/buffer.test.js

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Aug 15, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: ready for review.

Reproduced on bun 1.4.0 with the released binary: isUtf8(new DataView(ab)) and isAscii(new DataView(ab)) return true (node v26.3.0 throws ERR_INVALID_ARG_TYPE), and the rejection message for other inputs is First argument must be an ArrayBufferView instead of Node's text. The new cases in test/js/node/buffer.test.js (buffer.isUtf8() input handling / buffer.isAscii() input handling) fail 22/24 on 1.4.0 and pass 24/24 with this branch; the full file, Node's test-buffer-isutf8.js / test-buffer-isascii.js, and the utf-8-validate test also pass. Details and a node-vs-bun probe are in the PR description.

Self-review of the diff turned up nothing that needed changing; the one useful finding (Node main has since dropped the detached-ArrayBuffer throw in nodejs/node#64504, unreleased) is recorded in the description so the next upstream test port knows which two lines to update. CI: every lane that has run is green; the only test failures in the build are known flaky files unrelated to node:buffer that passed on retry, and two macOS shards are still queued for an agent.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — focused Node.js compat fix that consolidates two near-duplicate functions into one shared helper.

What was reviewed:

  • isTypedArrayType(view->type()) correctly excludes DataView; matches usage in BunObject.cpp/napi.cpp.
  • Removed impl() null check is safe — JSBuffer.cpp:827 already dereferences it unconditionally after the same downcast.
  • view->span() on a detached/out-of-bounds view is empty, so no memory is read; error helpers (INVALID_ARG_INSTANCE, throwError with ERR_INVALID_STATE) match existing call patterns.
  • utf-8-validate shim reuses jsBufferConstructorFunction_isUtf8 and now rejects DataView too — its test and documented API are Buffer-only, so unaffected.
Extended reasoning...

Overview

This PR fixes buffer.isUtf8() and buffer.isAscii() to match Node.js v26.3.0's input handling. It touches two files: src/jsc/modules/NodeBufferModule.h (~110 lines removed, ~50 added — two near-identical function bodies collapsed into one validateBytesOf template helper) and test/js/node/buffer.test.js (+86 lines of new describe.each coverage). The behavioral changes are: (1) reject DataView with ERR_INVALID_ARG_TYPE instead of scanning it, (2) use Node's exact error message via Bun::ERR::INVALID_ARG_INSTANCE for all rejected inputs, (3) return true for a TypedArray over a detached buffer instead of throwing, and (4) drop the Invalid state: prefix from the detached-ArrayBuffer error to match Node's C++-side wording.

Security risks

None. The only user-controlled input is the argument value, which is type-checked via JSC's dynamicDowncast before any bytes are read. Byte access goes through JSArrayBufferView::span() and ArrayBuffer::span(), both of which return an empty span for detached/out-of-bounds views, so there is no path to reading freed or out-of-range memory. No allocation, no size arithmetic, no user-callback reentrancy.

Level of scrutiny

Low-to-medium. This is a small, self-contained Node compat fix in a leaf function pair with no callers other than the module export table and the utf-8-validate third-party shim. The helper uses established in-tree idioms: isTypedArrayType() is the same predicate BunObject.cpp and napi.cpp use to distinguish TypedArray from DataView; INVALID_ARG_INSTANCE is the standard helper for class-instance error messages; the removed impl() null check matches JSBuffer.cpp:827 which already dereferences arrayBuffer->impl() unconditionally after the same downcast. The dropped RELEASE_AND_RETURN on the tail return is fine — neither jsBoolean nor simdutf::validate_* can throw, so there is no throwing tail call to release across.

Other factors

Test coverage is thorough: a describe.each over both functions asserting exact error class/code/message for DataView (three shapes), six rejected primitives/objects, detached ArrayBuffer, detached view (returns true), and every accepted input kind (all twelve TypedArrays, Buffer, ArrayBuffer, SharedArrayBuffer, sub-range view). The PR verified against Node v26.3.0 with a 49-input probe producing byte-identical output, and confirmed 22/24 new cases fail on bun 1.4.0. The ported Node tests test-buffer-isutf8.js / test-buffer-isascii.js and utf-8-validate.test.ts continue to pass. No outstanding reviewer comments; the only timeline entry is a CodeRabbit rate-limit notice.

@robobun

robobun commented Aug 15, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 12:13 PM PT - Aug 15th, 2026

🔄 @robobun, the build for your commit 2f7de0c2 (Build #98307) was cancelled — waiting for the next build...

Comment thread src/jsc/modules/NodeBufferModule.h Resolved
Comment thread src/jsc/modules/NodeBufferModule.h Resolved
Comment thread src/jsc/modules/NodeBufferModule.h Resolved

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — focused Node compat fix that collapses two duplicated argument-handling blocks into one helper and matches Node v26.3.0 exactly.

What was reviewed:

  • isTypedArrayType(view->type()) correctly excludes DataView (same predicate as BunObject.cpp/napi.cpp); INVALID_ARG_INSTANCE and Bun::throwError signatures/return types match.
  • Dropping the impl() null check is consistent with JSBuffer.cpp:827 and other callers that dereference it unconditionally.
  • view->span() on a detached view is safe (length 0, guarded by bytes.empty()) and is covered by the new detached-view test; the utf-8-validate shim sharing this function was checked — its documented arg is a Buffer.
  • Tests cover DataView (3 shapes), all TypedArray kinds, SAB, sub-range views, detached AB vs detached view, and exact error name/code/message for rejected inputs.
Extended reasoning...

Overview

Rewrites the argument handling for buffer.isUtf8() and buffer.isAscii() in src/jsc/modules/NodeBufferModule.h by replacing two ~50-line near-duplicate blocks with a single templated helper validateBytesOf. The behavioral changes: DataView is now rejected with ERR_INVALID_ARG_TYPE (was silently accepted because JSC's JSArrayBufferView includes DataView), rejected inputs get Node's exact The "input" argument must be an instance of ... message via Bun::ERR::INVALID_ARG_INSTANCE, a TypedArray over a detached buffer validates as empty (was throwing), and a detached ArrayBuffer throws ERR_INVALID_STATE without the Invalid state: prefix. Adds 86 lines of tests to test/js/node/buffer.test.js running both functions through the full input matrix via describe.each.

Security risks

None. This is input-type validation for a read-only byte scanner. No untrusted lengths drive allocation; span() returns JSC's bounds-checked view and the empty check runs before the simdutf call. No auth, crypto, or filesystem paths touched.

Level of scrutiny

Low-to-medium. Small, self-contained Node compat fix in a leaf function with no callers other than the module export and the utf-8-validate shim. The helper uses established in-tree primitives (isTypedArrayType, INVALID_ARG_INSTANCE, Bun::throwError) exactly as neighboring code does. I verified each helper's signature and return type in ErrorCode.h, and confirmed the impl() non-null assumption matches existing unconditional dereferences in JSBuffer.cpp, napi.cpp, and KeyObject.cpp. The dropped RELEASE_AND_RETURN is fine — nothing between scope declaration and the non-throwing return can raise an exception.

Other factors

Test coverage is thorough and asserts the strongest invariant (exact error name, code, and message; every TypedArray kind including Float16Array and BigInt arrays; SharedArrayBuffer; sub-range views; both detached forms). The PR description shows a 49-input probe with byte-identical output vs node v26.3.0, and reports the ported upstream test-buffer-isutf8.js/test-buffer-isascii.js and the utf-8-validate test all pass. The comment-cop bot's three inline notes were addressed in 2f7de0c (comments trimmed) and all threads are resolved. The bug hunting system found nothing.

@Jarred-Sumner Jarred-Sumner left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this use the possibly shared impl thing? Can it avoid cloning bytes

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants