Skip to content

node:zlib: honour rejectGarbageAfterEnd in the JS drivers and the gzip member loop - #47369

Open
robobun wants to merge 3 commits into
mainfrom
robobun/7c84a778/zlib-reject-garbage-after-end
Open

robobun wants to merge 3 commits into
mainfrom
robobun/7c84a778/zlib-reject-garbage-after-end

Conversation

@robobun

@robobun robobun commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • node:zlib ignores options.rejectGarbageAfterEnd: zlib.inflateSync(Buffer.concat([deflated, junk]), { rejectGarbageAfterEnd: true }) returns the data. Node.js v26.12.1 throws ERR_TRAILING_JUNK_AFTER_STREAM_END.
  • Nothing reads the option: ZlibBase (src/js/node/zlib.ts:246) only passes it to Transform.

Fix

  • As in Node's lib/zlib.js, ZlibBase validates and stores the option, and both drivers raise the error when a write leaves input. NativeZlib gets the flag at init and reads no second gzip member.
  • Correct: Node v26.12.1 passes the test file too (it uses node:test), and without the option no result changes (0 of 798 matrix cases).
  • Verified: test/js/node/zlib/reject-garbage-after-end.test.ts (124 pass, main fails 80). Also test/js/node/zlib/ and 68 vendored Node files.
  • Self-reviewed: 9 concerns raised, 4 addressed. The other 5 left with the zstd frame state.

Background

Downsides

  • zstd keeps no frame state yet. With the option, junk in a later write gives ZSTD_error_prefix_unknown, and 1 to 3 bytes of a frame magic pass.
  • Option unset, per stream: 1 more own property, at most 11 more bytecode instructions. Binary: +846 bytes (.text 256, built-in JS 590), one 4,096-byte page on disk.
  • As in Node, the option refuses valid input if finishFlush stops early (Z_BLOCK).
Notes

What Node.js v26.12.1 does

The option is in three places of Node:

  • lib/zlib.js: ZlibBase validates it and stores _rejectGarbageAfterEnd. processChunkSync throws after its loop. processCallback destroys the stream with the error.
  • src/node_zlib.cc, ZlibContext: with the flag, it does not go on to a second gzip member.
  • src/node_zlib.cc, ZstdDecompressContext: it keeps the state of the frames across writes and reports trailing input itself.

This PR has the first two. Node added the option in v26.5.0 (nodejs/node#64023).

Results

The script of the report runs inflate, inflateRaw, unzip, brotliDecompress and zstdDecompress, each in the sync and the callback form, on valid input with JUNK after it.

runtime forms that take the junk
main 10 of 10
this PR 0 of 10
Node.js v26.12.1 0 of 10

A matrix of 1327 cases ran on main, on this PR and on Node.js v26.12.1: 7 decoders, the option unset, true and false, the sync, callback and stream forms, and 21 inputs.

The matrix ran on the first commit of this PR. The second commit changes only how ZlibBase reads the option.

  • Node raises the error in 351 cases. This PR raises it in 328. main raises it in 0.
  • The other 23 cases are zstd. 18 are refused with ZSTD_error_prefix_unknown. 5 return the data.
  • 0 of the 798 cases without the option change.
  • In 5 more cases only bytesWritten differs from Node, because zlib-ng and Node's zlib give compressed data of different length.

zstd with the option set

input this PR Node.js v26.12.1
a frame and junk in one buffer the error the error
junk that starts a native write: a later write, or after a frame whose output fills the chunk ZSTD_error_prefix_unknown the error
a frame and 1 to 3 bytes of a frame magic number the data the error
a frame and a frame that is cut after its magic number the data Z_BUF_ERROR, with or without the option

Cost for a caller that does not set the option

Measured on release builds of the merge base (da867c6) and of this PR, linux-x64.

Each native write:

  • No instruction is added to a zlib, brotli or zstd write. The new test in the gzip member loop runs only when a gzip member ends with input left.

Each stream (a one-shot call makes one stream):

  • One more own property, _rejectGarbageAfterEnd, as in Node: 23 to 24 keys for a zlib class, 21 to 22 for brotli and zstd.
  • Reads of the option when an options object is given: 2 for a zlib class, 1 for brotli and zstd. Node: 3, 2 and 3.
  • Bytecode instructions that run at construction: ZlibBase 6 more with an options object and 4 more without. The Zlib constructor 4 more and 3 more. The Brotli and Zstd constructors 0.
  • Each *Sync call: 1 more bytecode instruction, the compare after the loop of processChunkSync. Each async write: 0. The new code of processCallback is inside the branch for input that is left.
  • GC cells for each call: no change. inflateSync 16.23, inflate with a callback 48.21. For 2000 streams the counts for each cell type are identical.
  • Property storage: capacity does not change, except for a Brotli stream in the stream form. Its out-of-line storage goes from 16 to 32 slots when it closes.

Each process:

  • .text: 108,801,193 to 108,801,449 bytes (+256). 163 of them are code in 4 functions: the init thunk +143, do_work_inflate +13, NativeZlibClass__construct +4, NativeZlibPrototype__init +3. The other 93 are padding.
  • .bun_builtins, the sources of the built-in JS modules: 2,748,332 to 2,748,922 bytes (+590).
  • .rodata, .data and .bss: no change.
  • The stripped binary: 138,270,840 to 138,274,936 bytes (+4,096, one page).
  • Bytecode of node:zlib: ZlibBase 173 to 184 instructions, Zlib 147 to 152, processChunkSync 180 to 192, processCallback 155 to 173. Brotli, Zstd and zlibOnError do not change.
  • size_of the Context of NativeZlib: 48 before and after. A const assertion holds it.
  • Host functions in GeneratedNodeZlibBinding.cpp: 22 before and after. One init thunk converts one more argument.

Differences from Node, on purpose

  • processCallback clears handle.buffer on the error path. Node leaves the chunk there. zlib.ts keeps buffer set only while a write is on the threadpool.
  • A last init argument that is not a boolean throws ERR_INVALID_ARG_TYPE. Node aborts the process.
  • ZlibBase reads the option one time, then validates and stores that value. Node reads it 2 times, and 3 times when it is set. The lint rule no-duplicate-conditional-property-access of this repository asks for the one read.

Not changed

  • NativeBrotli.init takes no flag, as in Node. The JS check covers brotli.
  • node:zlib/iter has no such option in Node.
  • DecompressionStream is native and does not use node:zlib. It keeps decoding a second gzip member (test/js/web/streams/compression.test.ts).
  • packages/bun-types: the option types of node:zlib come from @types/node. Version 26.6.5 declares the option.
  • Node's test-zlib-reject-garbage-after-end.js (v26.12.1) is not vendored, because a vendored file has to pass as a whole. 10 of its 12 tests pass with this PR, 3 of 12 on main. The other 2 need the zstd frame state, and one of the 2 asserts the zstd default.
  • test-zlib-type-error.js is not refreshed to v26.12.1. 3 of its 4 tests pass with this PR, 2 of 4 on main. Its gzip DecompressionStream case expects a TypeError for a second gzip member, and test/js/web/streams/compression.test.ts expects Bun to decode it.

The option on other paths

The check is "a write left input and had output space". Two more paths reach it, and Node.js v26.12.1 gives the same result for both:

  • inflateSync(valid, { finishFlush: Z_BLOCK, rejectGarbageAfterEnd: true }) is the error.
  • A zstd compressor with nbWorkers can leave input in a write. With the option that is the error. Without the option the stream ends with no output, on main too.

Found on the way, not in this PR

  • A second init() on a NativeZlib handle leaks the first zlib stream. 5000 calls take main from 36 MiB to 1037 MiB. It has its own fix in progress.
  • stream._handle.onerror reads back as an internal object inside AsyncLocalStorage.run. It has its own fix in progress.
  • The test for the native init argument makes a new handle for each call, so it does not reach that leak.

Test runs

  • bun bd test test/js/node/zlib/reject-garbage-after-end.test.ts: 124 pass, 6 todo, 0 fail. With --todo the 6 still fail. With the environment of the ASAN lane (leak check and exception check): exit 0.
  • The same file on Node.js v26.12.1 (node --test): 116 pass, 6 todo, 0 fail. With each todo turned into a test: 122 pass.
  • The same file on main: 44 pass, 6 todo, 80 fail.
  • The file uses node:test and node:assert, so Node runs the identical file and both results are above. zlib.test.js imports bun:test, so this is a new file.

Self-review

The review raised 9 concerns.

  • Addressed here: the test for the native init argument called init on a handle that was already initialized (now a new handle for each call), a test title said that compressors ignore the option (the title and this body now say what the check is), the zstd cases that stay open were not listed or tested (now todo tests, one live test and the table above), and the divergence for handle.buffer had no comment.
  • Moved out with the zstd frame state: the wording of the default change, 6 clauses of the frame code without a test, reset() of the frame state without a test, an instruction count that was true only inside a frame, and the effect of the default change on minizlib.

[human-review] gate passed · iteration 1 · 6 files touched

fails on main (without fix)
ASAN without fix: 80 failed, 6 skipped
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" test/js/node/zlib/reject-garbage-after-end.test.ts
bun test v1.4.4 (367d939d9)

test/js/node/zlib/reject-garbage-after-end.test.ts:
(pass) rejectGarbageAfterEnd: true > inflate > decodes input that ends with the stream [269.53ms]
104 |     });
105 |   });
106 | }
107 | 
108 | async function assertRefused(decoder: Decoder, input: Buffer, options: Options) {
109 |   assert.throws(() => decoder.sync(input, options), trailingJunk);
               ^
AssertionError: Missing expected exception (TypeError).
 generatedMessage: false,
     actual: undefined,
   expected: {
  name: "TypeError",
  code: "ERR_TRAILING_JUNK_AFTER_STREAM_END",
  message: "Trailing junk found after the end of the compressed stream",
},
   operator: "throws",
       diff: "simple",
       code: "ERR_ASSERTION"

      at assertRefused (/workspace/bun/test/js/node/zlib/reject-garbage-after-end.test.ts:109:10)
      at /workspace/bun/test/js/node/zlib/reject-garbage-after-end.test.ts:137:15
      at run (node:async_hooks:229:29)
      at node:test:1781:26
      at executeT
... (truncated)

release without fix: 6 skipped
bun test v1.4.4-canary.1 (810d86515)

test/js/node/zlib/reject-garbage-after-end.test.ts:
(pass) rejectGarbageAfterEnd: true > inflate > decodes input that ends with the stream [37.98ms]
(pass) rejectGarbageAfterEnd: true > inflate > refuses bytes that are not a compressed stream [3.12ms]
(pass) rejectGarbageAfterEnd: true > inflate > refuses zero padding [1.86ms]
(pass) rejectGarbageAfterEnd: true > inflate > refuses one more byte [1.47ms]
(pass) rejectGarbageAfterEnd: true > inflate > refuses a second compressed stream [0.76ms]
(pass) rejectGarbageAfterEnd: true > inflate > refuses trailing input after output that fills the chunk exactly [2.38ms]
(pass) rejectGarbageAfterEnd: true > inflate > refuses trailing input with info: true [0.27ms]
(pass) rejectGarbageAfterEnd: true > inflate > the sync form closes the handle before it throws [0.32ms]
(pass) rejectGarbageAfterEnd: true > inflate > a stream gives what it decoded, then the error, and no 'end' [2.26ms]
(pass) rejectGarbageAfterEnd: true > inflate > reset() keeps the option [0.60ms]
(pass) rejectGarbageAfterEnd: true > inflateRaw > decodes input that ends with the stream [2.26ms]
(pass) rejectGarbageAfterEnd: 
... (truncated)
passes on PR (with fix)
ASAN with fix: 6 skipped
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" test/js/node/zlib/reject-garbage-after-end.test.ts
bun test v1.4.4 (367d939d9)

test/js/node/zlib/reject-garbage-after-end.test.ts:
(pass) rejectGarbageAfterEnd: true > inflate > decodes input that ends with the stream [307.18ms]
(pass) rejectGarbageAfterEnd: true > inflate > refuses bytes that are not a compressed stream [89.39ms]
(pass) rejectGarbageAfterEnd: true > inflate > refuses zero padding [100.20ms]
(pass) rejectGarbageAfterEnd: true > inflate > refuses one more byte [44.38ms]
(pass) rejectGarbageAfterEnd: true > inflate > refuses a second compressed stream [24.57ms]
(pass) rejectGarbageAfterEnd: true > inflate > refuses trailing input after output that fills the chunk exactly [112.81ms]
(pass) rejectGarbageAfterEnd: true > inflate > refuses trailing input with info: true [11.14ms]
(pass) rejectGarbageAfterEnd: true > inflate > the sync form closes the handle before it throws [13.06ms]
(pass) rejectGarbageAfterEnd: true > inflate > a stream gives what it decoded, then the error, and no 'end' [45.21ms]
(pass) rejectGarbageAfter
... (truncated)

release with fix: 6 skipped
$ bun scripts/build.ts --profile=release
[configured] bun-profile → bun (stripped) in 3647ms (unchanged)
ninja: Entering directory `/workspace/bun/build/release'
[1/414] build.rs build_script_build
[2/414] gen ErrorCode+*.h
[3/414] gen cpp.rs (cppbind)
[4/414] gen generated_host_exports.rs
generated_host_exports.rs: 111 exports (lazy=4, generic=107, rust=0); 260 extern-C blocks audited
[5/413] rustc bun_safety 
[6/413] rustc bun_zlib_sys 
[7/413] rustc bun_output 
[8/413] rustc bun_perf 
[9/413] rustc bun_ptr 
[10/413] rustc bun_zstd 
[11/413] rustc bun_boringssl_sys 
[12/413] rustc bun_base64 
[13/413] rustc bun_cares_sys 
[14/413] rustc bun_picohttp 
[15/413] rustc bun_brotli 
[16/413] rustc bun_clap 
[17/413] rustc bun_lsquic_sys 
[18/413] rustc bun_errno 
[19/413] rustc bun_text_diff 
[20/413] rustc bun_tcc_sys 
[21/413] rustc bun_valkey 
[22/413] gen the table of built-ins
builtins table: operands without uses: none
builtins table: 2011 host entries have a result that is not described; a program in which a value may be anything holds what any host class needs: 0 modules, 15 functions
[23/400] rustc bun_core 
[24/400] rustc bun_paths 
[25/400] rustc bun_shell_
... (truncated)
diff hotspot
src/js/builtins.d.ts                               |   1 +
 src/js/node/zlib.ts                                |  44 ++-
 src/js/private.d.ts                                |   1 +
 src/runtime/node/node_zlib_binding.bind.ts         |   1 +
 src/runtime/node/zlib/NativeZlib.rs                |  14 +-
 test/js/node/zlib/reject-garbage-after-end.test.ts | 432 +++++++++++++++++++++
 6 files changed, 488 insertions(+), 5 deletions(-)

gate history · 2 passed · 0 rejected · iteration 1

evidence per changed file
file                                                reads  edits  tests
src/js/builtins.d.ts                                    0      0     37
src/js/node/zlib.ts                                     3      4     38
src/js/private.d.ts                                     0      0     37
src/runtime/node/node_zlib_binding.bind.ts              0      0     36
src/runtime/node/zlib/NativeZlib.rs                     0      0     31
test/js/node/zlib/reject-garbage-after-end.test.ts      1      2     30

`options.rejectGarbageAfterEnd` was accepted and ignored. As in Node.js
v26.12.1:

- ZlibBase validates the option as a boolean and stores it.
- processChunkSync and processCallback raise
  ERR_TRAILING_JUNK_AFTER_STREAM_END when a native write leaves input and
  has output space. Every decoder goes through one of the two.
- NativeZlib takes the flag at init. With the flag it does not read a
  second gzip member.

No result changes for a caller that does not set the option.

The zstd decoder keeps no frame state across writes yet. With the option
it refuses what one write leaves. Bytes that come in a later write still
give ZSTD_error_prefix_unknown, and 1 to 3 bytes of a frame magic number
are still accepted.
@robobun

robobun commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status

Reproduced on main: zlib.inflateSync(Buffer.concat([zlib.deflateSync("hello"), Buffer.from("JUNK")]), { rejectGarbageAfterEnd: true }) returns hello. The same happens for inflate, inflateRaw, unzip, brotliDecompress and zstdDecompress, in the sync and the callback form: 10 of 10 take the junk. Node.js v26.12.1 throws ERR_TRAILING_JUNK_AFTER_STREAM_END in all 10.

With this PR (#47369) all 10 forms throw that error.

CI (build 135313, last commit): 186 of 188 jobs ran, and all 186 passed. These are all builds, all Linux, Windows and macOS aarch64 test lanes with the ASAN lane, and 2 of the 4 macOS x64 test jobs. The other 2 macOS x64 test jobs never ran. They waited for an agent longer than BuildKite allows and expired 4 times, and that is why the build shows as failed. A retry of these 2 jobs completes the build. The new test file is in one of them, so macOS x64 is the one platform where it did not run yet. No failure comes from this change.

@robobun

robobun commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 1:26 PM PT - Oct 29th, 2026

❌ @robobun, your commit 36eab97 has some failures in Build #135313 (All Failures)


🧪   To try this PR locally:

bunx bun-pr 47369

That installs a local version of the PR into your bun-47369 executable, so you can run:

bun-47369 --bun

ZlibBase read the option in a condition and again in the body of that
condition. The lint rule no-duplicate-conditional-property-access does
not allow that. ZlibBase now reads the option into a local, validates
the local and stores it.
@robobun
robobun marked this pull request as ready for review October 29, 2026 14:32
@robobun
robobun requested a review from alii as a code owner October 29, 2026 14:32
@coderabbitai

coderabbitai Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 1 minute.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
Commits

Reviewing files that changed from the base of the PR and between da867c6 and 36eab97.

Files selected for processing: 6
  • src/js/builtins.d.ts
  • src/js/node/zlib.ts
  • src/js/private.d.ts
  • src/runtime/node/node_zlib_binding.bind.ts
  • src/runtime/node/zlib/NativeZlib.rs
  • test/js/node/zlib/reject-garbage-after-end.test.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Comment thread src/js/node/zlib.ts Resolved

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants