Skip to content

node:stream: install the Readable operators with the class instead of the node:stream module - #37395

Open
robobun wants to merge 3 commits into
mainfrom
farm/2927157e/readable-operators-without-stream-umbrella
Open

robobun wants to merge 3 commits into
mainfrom
farm/2927157e/readable-operators-without-stream-umbrella

Conversation

@robobun

@robobun robobun commented Aug 11, 2026 •

Copy link
Copy Markdown
Collaborator

What does this PR do?

Readable.prototype.map/filter/flatMap/drop/take/forEach/toArray/some/every/find/reduce were missing from any stream reached without require("node:stream"):

$ bun -e 'const net = require("node:net"); console.log(typeof new net.Socket().map)'
undefined
$ bun -e 'const net = require("node:net"); require("node:stream"); console.log(typeof new net.Socket().map)'
function
$ node -e 'const net = require("node:net"); console.log(typeof new net.Socket().map)'
function

Same for crypto.createHash() (and Hmac/Cipheriv, via LazyTransform), child process stdio streams, and the _stream_readable / _stream_duplex / _stream_transform / _stream_passthrough modules. Whether socket.toArray() worked depended on whether something else in the process happened to have loaded node:stream first.

Cause. The operator wrappers were installed on Readable.prototype by src/js/internal/stream.ts, the module behind node:stream. That mirrors Node's lib/stream.js, but in Node every internal user of the class goes through require('stream'), so the install always runs before user code can get hold of a Readable. In Bun, net, tls, crypto, child_process, worker_threads, _http_incoming, internal/streams/native-readable and the public _stream_* modules require internal/streams/readable / internal/streams/duplex directly (on purpose, to avoid loading the whole umbrella at startup), so the class existed without its operators.

Fix. Install the operators at the end of internal/streams/readable.ts, where the class is defined, and drop the loops from internal/stream.ts. Loading the class is now what completes the prototype, so it no longer matters which module reaches it first, and future direct consumers (for example #27400, which would route fs streams and zlib around the umbrella as well) stay correct without having to know about this. internal/streams/operators only depends on validators, shared and end-of-stream, all of which readable.ts already loads, so this introduces no require cycle; the cost is that loading the class now also evaluates operators.ts itself (the eleven operator functions, about 400 lines, no further requires), which is what require("node:stream") was already paying and is what #35541 settled on too. The wrappers themselves are unchanged except that they call Readable.from directly instead of going through Stream.Readable at call time (Readable.from is still looked up when the operator runs); name, length and the property attributes are the same as before and match Node.

#35541 contains this same move as one piece of a much larger lazy-loading change (it needs it, because it makes Stream.Readable itself lazy). This PR is only the user-visible fix plus a test so it can land on its own; that PR would drop its copy of the block when rebased.

How did you verify your code works?

Added tests to test/js/node/stream/node-stream.test.js. Each spawns a fresh bun -e that never requires node:stream (the script is wrapped in a block, because -e exposes the builtin modules as globals and a top-level const stream would be enough to load node:stream and mask the bug): a table checks that all eleven operators are functions on net.Socket, crypto.createHash(), a child process stdout and the prototypes exported by the four _stream_* modules, and two more tests actually run map().toArray() on a connected net.Socket and filter().map().toArray() on a _stream_readable stream. All nine fail on the current release (USE_SYSTEM_BUN=1: every operator is undefined, socket.map is not a function) and pass with this change.

With the debug build I also ran the rest of test/js/node/stream/ and the vendored Node tests for the operators (test-stream-map, -filter, -flatMap, -forEach, -reduce, -toArray, -drop-take, -some-find-every, -compose, -compose-operator, -duplex-from, -iterator-helpers-test262-tests) plus a handful of crypto hash/stream, child_process stdio, net, tls and worker stdio tests; all pass.


no test proof · iteration 0 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/node/stream/node-stream.test.js

…lla module

The map/filter/forEach/toArray/... operators were defined on
Readable.prototype by internal/stream.ts, so any Readable reached without
requiring node:stream (net.Socket, tls, crypto hashes, child_process stdio,
_stream_readable and friends) lacked them until something else loaded the
umbrella module. Install them at the end of internal/streams/readable.ts so
every Readable has them as soon as the class exists.
@coderabbitai

coderabbitai Bot commented Aug 11, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Changes

Readable operator registration

Layer / File(s) Summary
Move operator registration to Readable
src/js/internal/stream.ts, src/js/internal/streams/readable.ts
Operator methods are no longer registered by internal/stream.ts. Readable initialization now installs stream-returning and promise-returning methods with constructor checks and result dispatch.
Validate cross-module operator availability
test/js/node/stream/node-stream.test.js
Subprocess tests verify operator availability across stream-related modules and validate map, filter, and toArray behavior.

Suggested reviewers: cirospaciari

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes moving Readable operator installation from node:stream to the Readable class module.
Description check ✅ Passed The description includes both required sections and provides detailed context, implementation scope, and verification results.

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Aug 11, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: reproduced on bun 1.4.0 (bun -e 'const net = require("node:net"); console.log(typeof new net.Socket().map)' prints undefined, node prints function; same for crypto.createHash(), child process stdio and the _stream_* modules). The nine new tests in test/js/node/stream/node-stream.test.js fail on the release build and pass with this branch (re-checked after the two follow-up commits: block-scoped -e scripts, shorter comment). Waiting on CI.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — straightforward relocation of the operator-install loops from internal/stream.ts to the end of internal/streams/readable.ts.

What was reviewed:

  • Verified internal/streams/operators only requires validators, shared and end-of-stream, all of which readable.ts already loads — no new require cycle.
  • Checked the moved block is byte-identical to the original except Stream.Readable.from → Readable.from and the loop-variable rename to avoid shadowing streamKeys used in wrap().
  • Confirmed ObjectKeys was the only removed helper in stream.ts and is already defined in readable.ts; no dead constants left behind.
  • Tests spawn fresh processes with piped stdout/stderr drained concurrently, use listen(0), and cover both operator groups end-to-end.
Extended reasoning...

Overview

The PR moves two for loops that install the eleven Readable operator wrappers (map/filter/flatMap/drop/take and every/forEach/reduce/toArray/some/find) from src/js/internal/stream.ts (the node:stream umbrella) to the bottom of src/js/internal/streams/readable.ts (where the Readable class is defined). The umbrella module drops the loops and its now-unused ObjectKeys/operators imports. Nine subprocess tests are added to test/js/node/stream/node-stream.test.js.

Security risks

None. This is a pure module-load-order fix in built-in JS; no user input parsing, no auth/crypto/permissions surface. The wrapper bodies (including the new.target guard, __proto__: null descriptors, and ``) are unchanged.

Level of scrutiny

Low-to-medium. The moved code is verbatim modulo two mechanical renames: Stream.Readable.from becomes the local Readable.from (defined ~20 lines above the new block, and still looked up at call time so user monkey-patching semantics are preserved), and streamKeys/promiseKeys become opStreamKeys/opPromiseKeys to avoid shadowing the existing streamKeys local inside Readable.prototype.wrap. The main risk with this kind of move is a require cycle; I checked internal/streams/operators.ts and it only pulls in internal/validators, internal/shared and internal/streams/end-of-stream, all three of which readable.ts already required at its top, so the dependency graph is unchanged.

Other factors

The tests follow the harness conventions well: it.concurrent.each over the seven affected entry points, subprocess pipes drained via Promise.all, a combined { stdout, stderr, exitCode } assertion, listen(0, "127.0.0.1") for the net test, and no external network or sleeps. The PR description explains why this diverges from Node's lib/stream.js placement (Bun's internal consumers bypass the umbrella on purpose) and notes the interaction with #35541. The bug hunter found nothing. This is small, self-contained, and well-covered — safe to approve without a human pass.

Comment thread src/js/internal/streams/readable.ts Outdated
@robobun

robobun commented Aug 11, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 2:13 AM PT - Aug 11th, 2026

✅ @robobun, your commit 98c96378ef3bd9206135b95c9e9bd9ba51d7bc2b passed in Build #92109! 🎉


🧪   To try this PR locally:

bunx bun-pr 37395

That installs a local version of the PR into your bun-37395 executable, so you can run:

bun-37395 --bun

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — this is a straight relocation of the existing operator-install loops from internal/stream.ts to the tail of internal/streams/readable.ts, so the class is complete regardless of which module reaches it first.

What was reviewed

  • Verified internal/streams/operators only requires validators/shared/end-of-stream (all already loaded by readable.ts) — no require cycle.
  • Wrapper bodies are byte-identical to the removed ones except Stream.Readable.from → Readable.from (same object; still looked up at call time), and Readable.from is defined earlier in the file.
  • ObjectKeys/ObjectDefineProperty are both in scope; the opStreamKeys rename avoids reusing the streamKeys local from wrap().

The comment-cop bot flag on line 1729 looks like a false positive — that's a two-line note explaining why the install location diverges from Node's lib/stream.js, not a workaround justification.

Extended reasoning...

Overview

Moves the two loops that install map/filter/flatMap/drop/take/forEach/toArray/some/every/find/reduce on Readable.prototype from src/js/internal/stream.ts (the node:stream umbrella) to the end of src/js/internal/streams/readable.ts (where Readable is defined). The umbrella now just requires internal/streams/readable and gets a fully-populated prototype. Nine subprocess tests are added to test/js/node/stream/node-stream.test.js covering net.Socket, crypto.createHash(), child-process stdout, and the four _stream_* module prototypes, plus two functional tests that actually run the operators.

Security risks

None. No user-controlled input parsing, no auth/crypto/permission surface. The operator wrappers themselves are unchanged (still use ., __proto__: null descriptors, and new.target guards), and no validation was weakened.

Level of scrutiny

Low-to-medium. This touches the widely-used node:stream compat layer, but the change is a mechanical cut-and-paste of ~40 lines. The only semantic delta is Stream.Readable.from(...) → Readable.from(...), which is the same function on the same object. I verified internal/streams/operators has no dependency on readable (it only requires validators, shared, end-of-stream), so moving the require earlier introduces no cycle. ObjectDefineProperty is newly captured at module top; ObjectKeys was already there for wrap().

Other factors

  • Tests are hermetic (spawn fresh bun -e processes, block-scope the script to avoid bun -e's builtin globals accidentally loading node:stream), use it.concurrent, drain both pipes with Promise.all, and assert the full {stdout, stderr, exitCode} object. The PR description confirms they fail under USE_SYSTEM_BUN=1 and pass with the change.
  • The removed ObjectKeys and streamReturningOperators/promiseReturningOperators imports from stream.ts have no other references in that file.
  • The github-actions comment-cop bot flagged the two-line comment above the new block. That comment explains why Bun installs here rather than mirroring Node's lib/stream.js — it's short, factual, and answers the "why not the alternative" question the review guidelines call for. Not a workaround; not a blocker.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — moves the existing operator-install loops from internal/stream.ts to the end of internal/streams/readable.ts so the prototype is complete regardless of which module loads it first.

What was reviewed:

  • Confirmed internal/streams/operators only requires validators, shared and end-of-stream, so no cycle is introduced by requiring it from readable.ts.
  • Wrapper bodies are byte-identical to the removed ones except Stream.Readable.from → Readable.from (same function, still looked up at call time).
  • ObjectKeys was already in scope in readable.ts; ObjectDefineProperty alias is added alongside the existing ObjectDefineProperties.
  • Tests spawn fresh block-scoped -e processes so node:stream never loads, cover all eleven operators across seven entry points, and functionally exercise map/filter/toArray.
Extended reasoning...

Overview

The PR relocates two for loops (~40 lines) that install the eleven Readable stream operators (map, filter, flatMap, drop, take, forEach, toArray, some, every, find, reduce) on Readable.prototype. Previously they ran in src/js/internal/stream.ts (the module behind node:stream); now they run at the bottom of src/js/internal/streams/readable.ts where Readable is defined. The umbrella module drops the loops and its now-unused ObjectKeys alias and operators require. A new describe block in test/js/node/stream/node-stream.test.js adds nine subprocess tests.

Security risks

None. This is a pure code move within built-in JS modules — no user input handling, no native code, no new API surface. The wrapper functions themselves are unchanged.

Level of scrutiny

Low-to-medium. The diff is small and mechanical: the two loops are copied verbatim, with Stream.Readable.from replaced by the local Readable.from (identical target) and the loop variable names renamed to avoid shadowing the existing streamKeys local in Readable.prototype.wrap. The main risk with a move like this is a require cycle, and I verified internal/streams/operators depends only on internal/validators, internal/shared and internal/streams/end-of-stream — all three are already required by readable.ts above the new block, so evaluation order is unchanged.

Other factors

  • ObjectKeys and ObjectDefineProperty are both in scope in the new location (the former already existed for wrap(), the latter is added).
  • The property descriptors (name, length, non-enumerable/configurable/writable) are preserved exactly, so Node compat for Object.getOwnPropertyDescriptor(Readable.prototype, 'map') etc. is unchanged.
  • Tests follow harness conventions: bunEnv/bunExe, concurrent draining of stdout/stderr/exited, it.concurrent.each for the matrix, port 0 for the net test, and a combined-object toEqual assertion. The block-scoping of -e scripts (to avoid Bun's lazy builtin globals loading node:stream) is documented and was iterated on in commit 3eb269e.
  • The comment-cop bot's inline note about a long comment was addressed in 98c9637 (shortened to one line) and the thread is resolved.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant