Skip to content

fetch, node:fs: take PathBuffer scratch from the pool in fetch_impl and NodeFS - #40680

Closed
robobun wants to merge 6 commits into
mainfrom
farm/f62aaad7/fetch-stack-frame
Closed

robobun wants to merge 6 commits into
mainfrom
farm/f62aaad7/fetch-stack-frame

Conversation

@robobun

@robobun robobun commented Aug 28, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • Every fetch() on Windows x64 reserves about 297 KB of native stack (the fetch_impl<false> prologue in 1.4.0: movl $0x48bb8, %eax; callq __chkstk). Linux: 11 KB.
  • fetch_impl (src/runtime/webcore/fetch.rs) holds five PathBuffer-sized locals, three live at once, and a PathBuffer is 98 302 bytes on Windows. One of the five is a NodeFS, which embeds a PathBuffer by value, so every NodeFS::default() stack local in the tree (15 sites) is 96 KB too.
  • A body whose Symbol.asyncIterator getter calls fetch() again recurses 16 levels on Windows before the RangeError, 387 on Linux.

Fix

  • The four PathBuffer locals come from bun_paths::path_buffer_pool::get(): a 16-byte guard on the stack, the buffer on the heap.
  • NodeFS::sync_error_buf becomes a path_buffer_pool::Guard. NodeFS shrinks to two words and every NodeFS::default() site with it. #[repr(C)] goes: the u16 reinterpretation now relies on the heap allocation's alignment, which the existing assert! checks.
  • Correct because each buffer is write-only scratch that does not outlive its owner, and the pool hands out the same PathBuffer type. A probe over file:, blob: and Bun.file() bodies (Notes) gives identical output before and after.
  • Verified: test/js/web/fetch/body-async-iterator.test.ts, which stock 1.4.1 fails on Windows (16 < 32). Windows debug build: 9 levels before, 96 after. Windows-host clippy large_stack_frames (128 KB threshold): 45 functions before, 38 after.

Background

  • PathBuffer is [u8; MAX_PATH_BYTES]: 4096 bytes on Linux, 1024 on macOS, 98 302 on Windows.
  • bun_paths::path_buffer_pool caches up to four heap PathBuffers per thread behind an RAII guard, for exactly this problem.
  • A function reserves the stack slots of all its locals in its prologue, taken branch or not. On Windows __chkstk then probes every page of the frame: 73 pages per fetch() call here.
  • JSC checks its 5 MB stack budget in JS prologues. Bun's main and worker threads reserve 18 MB on Windows, so the big frame costs per-call work and shallow re-entrant recursion, not an overflow.
Notes
  • The five locals: path_buf, path_buf2, cwd_buf (Windows only) in the file:/blob: branch, open_path_buf and NodeFS::sync_error_buf in the Bun.file() body branch. The two branches never run together, so the compiler shares their slots and the frame holds three buffers. On Linux cwd_buf does not exist: 2 x 4 KB plus 3 KB of other locals matches the 11 KB prologue.
  • The test cannot fail on Linux on the unfixed build: PathBuffer is 4 KB there, so the recursion already reaches 387 levels (release) or 57 (debug, ASAN). It fails on Windows only. Measured on windows-x64 with stock 1.4.1-canary: Expected: >= 32, Received: 16. With this branch, the Windows debug build reaches 96 and the Linux debug build 71. The test passed on every CI lane, including both Windows release lanes.
  • Threshold: 32 is twice the unfixed Windows number and a third of the fixed Windows debug number. Release builds have smaller frames and reach more levels.
  • The test uses a pre-aborted AbortSignal, so the recursion happens in body extraction and no socket is opened (strace -e connect shows zero calls).
  • clippy numbers: cargo clippy -p bun_runtime --no-deps on a Windows host (large_stack_frames is deny in Cargo.toml with stack-size-threshold = 131072, but CI runs clippy on Linux only; bun_core: fix clippy on the Windows and FreeBSD targets and lint them in CI #37605 adds the Windows target). Functions that leave the list: fetch.rs:385 fetch_impl (539 424 bytes estimated, clippy does not model slot sharing), node_fs.rs:4427 NodeFS::default, node_fs_binding.rs:126 Binding::new, jsc_hooks.rs:1323 create_node_fs, Blob.rs:4285, test_command.rs:1668, aggregate.rs:164. The 38 that remain are mostly CLI commands; the JS-facing ones are Listener.rs:176 (Bun.listen, 206 KB), path.rs:1355 (path.resolve, 197 KB), node_fs.rs:8258 (297 KB), copy_file.rs:1475 (199 KB), glob.rs:33 (296 KB) and filesystem_router.rs:109 (203 KB). Those are follow-ups.
  • Why not the per-VM NodeFS (VirtualMachine::node_fs()) for the fetch read: when NodeFS.vm is set (standalone executables), read_file claims the VM pipe-read scratch and returns a MarkedArrayBuffer created in the JSC heap. The fetch caller copies result.slice() and calls destroy(), which is a no-op for that buffer, so it would work but leave a dead ArrayBuffer per Bun.file() body. A fresh NodeFS with vm: None is what the code did before, and it is now cheap.
  • Pre-existing and unchanged by this PR: on a Windows debug build, fetch("file:rel.txt") panics at debug_assert!(crate::is_absolute_windows(maybe_posix_path)) in resolve_cwd_with_external_buf_z. The URL parser turns the input into the path /rel.txt, is_absolute accepts the leading slash, then the Windows branch strips it and passes the relative rel.txt to the normalizer. Release builds skip the assertion and open the file relative to cwd. Reproduced on main.
  • Suites run: fetch.test.ts (same 25 release / 41 debug environment failures before and after, all Unable to connect to a localhost listener or debug timeouts), fetch-file-upload.test.ts, blob.test.ts, fetch-compress.test.ts, body.test.ts, fetch-args.test.ts, regression/issue/29787.test.ts, node/fs/{fs,fs-mkdir,fs-path-length,cp,promises}.test, bun-write.test.js, bun-file.test.ts, shell/commands/mkdir.test.ts, blob-write.test.ts, test/internal/source-lints. On Windows: body-async-iterator, fs, fs-mkdir, fs-path-length, readdir-windows-ntstatus, rm-windows-ntstatus, bun-file-windows, bun-write. The one failure seen (readSync > works on large files, 5 s timeout) also times out with the stock canary on that machine.
  • Probe script (run with both builds, output diffed on Linux and Windows):
await fetch(pathToFileURL(dir + "/rel.txt").href);            // abs
await fetch(href.replace("rel.txt", "rel%2Etxt"));            // percent-decoded
await fetch(pathToFileURL(dir).href);                         // EISDIR
await fetch(URL.createObjectURL(new Blob(["blob-body"])));    // blob:
await fetch(url, { method: "POST", body: Bun.file(small) });  // read_file path
await fetch(url, { method: "POST", body: Bun.file(big64k) }); // sendfile path
await fetch(url, { method: "POST", body: Bun.file(big64k).slice(100, 1100) });
await fetch(url, { method: "POST", body: Bun.file(missing) }); // ENOENT
await fetch(url, { method: "POST", body: Bun.file(openSync(small, "r")) }); // fd

[review] gate passed · iteration 0 · 4 files touched

fails on main (without fix)
ASAN without fix: BUILD FAILED (no junit output)
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/web/fetch/body-async-iterator.test.ts
ninja: Entering directory `/workspace/bun/build/debug'
[1/167] gen generated_host_exports.rs
generated_host_exports.rs: 122 exports (host=5, lazy=10, generic=107, rust=0); 243 extern-C blocks audited
[2/167] gen JSSink.{cpp,h,lut.h,rs}
generated_jssink.rs: 7 sinks, 84 exported symbols
Generating /workspace/bun/build/debug/codegen/JSSink.lut.h from /workspace/bun/build/debug/codegen/JSSink.lut.txt
[3/167] gen ZigGeneratedClasses.{cpp,h,rs}
Found 2 classes from /workspace/bun/src/jsc/resolve_message.classes.ts
  - ResolveMessage (15 fields)
  - BuildMessage (10 fields)
Found 1 classes from /workspace/bun/src/runtime/api/Archive.classes.ts
  - Archive (4 fields, 1 class fields)
Found 2 classes from /workspace/bun/src/runtime/api/BunObject.classes.ts
  - ResourceUsage (8 fields)
  - Subprocess (20 fields)
Found 1 classes from /workspace/bun/src/runtime/api/cron.classes.ts
  - CronJob (5 fields)
Found 3 classes from /workspace/bun/src/runtime/api/filesystem_router.classes.ts
  - FileSystemRouter
... (truncated)

release without fix: all passed
bun test v1.4.1-canary.1 (d699f46e0)

test/js/web/fetch/body-async-iterator.test.ts:
(pass) Response.bytes() with async iterable body does not crash with null deref [8.22ms]
(pass) fetch() called from a body's Symbol.asyncIterator getter recurses deeply before the stack limit [14.42ms]
(pass) Response.arrayBuffer() with async iterable body does not crash with null deref [8.96ms]

 3 pass
 0 fail
 7 expect() calls
Ran 3 tests across 1 file. [142.00ms]
__F:0:S:0
passes on PR (with fix)
ASAN with fix: all passed
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/web/fetch/body-async-iterator.test.ts
bun test v1.4.1 (65362b53b)

test/js/web/fetch/body-async-iterator.test.ts:
(pass) Response.bytes() with async iterable body does not crash with null deref [429.16ms]
(pass) fetch() called from a body's Symbol.asyncIterator getter recurses deeply before the stack limit [510.71ms]
(pass) Response.arrayBuffer() with async iterable body does not crash with null deref [446.47ms]

 3 pass
 0 fail
 7 expect() calls
Ran 3 tests across 1 file. [4.33s]
__F:0:S:0

release with fix: all passed
$ bun scripts/build.ts --profile=release
[configured] bun-profile → bun (stripped) in 877ms (unchanged)
ninja: Entering directory `/workspace/bun/build/release'
[1/127] gen generated_host_exports.rs
generated_host_exports.rs: 122 exports (host=5, lazy=10, generic=107, rust=0); 243 extern-C blocks audited
[2/127] gen ZigGeneratedClasses.{cpp,h,rs}
Found 2 classes from /workspace/bun/src/jsc/resolve_message.classes.ts
  - ResolveMessage (15 fields)
  - BuildMessage (10 fields)
Found 1 classes from /workspace/bun/src/runtime/api/Archive.classes.ts
  - Archive (4 fields, 1 class fields)
Found 2 classes from /workspace/bun/src/runtime/api/BunObject.classes.ts
  - ResourceUsage (8 fields)
  - Subprocess (20 fields)
Found 1 classes from /workspace/bun/src/runtime/api/cron.classes.ts
  - CronJob (5 fields)
Found 3 classes from /workspace/bun/src/runtime/api/filesystem_router.classes.ts
  - FileSystemRouter (5 fields)
  - FrameworkFileSystemRouter (2 fields)
  - MatchedRoute (8 fields)
Found 1 classes from /workspace/bun/src/runtime/api/Glob.classes.ts
  - Glob (5 fields)
Found 1 classes from /workspace/bun/src/runtime/api/h2.classes.ts
  - H2FrameParser (32 fields)
Found 
... (truncated)
diff hotspot
src/runtime/jsc_hooks.rs                      |  2 +-
 src/runtime/node/node_fs.rs                   | 23 +++++-------------
 src/runtime/webcore/fetch.rs                  | 18 +++++---------
 test/js/web/fetch/body-async-iterator.test.ts | 35 +++++++++++++++++++++++++++
 4 files changed, 48 insertions(+), 30 deletions(-)

gate history · 1 passed · 0 rejected · iteration 0

evidence per changed file
file                                           reads  edits  tests
src/runtime/jsc_hooks.rs                           1      2      0
src/runtime/node/node_fs.rs                       10      7      0
src/runtime/webcore/fetch.rs                       9      9      0
test/js/web/fetch/body-async-iterator.test.ts      1      1      0

@robobun

robobun commented Aug 28, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: fix and test pushed. The PR now also stores NodeFS::sync_error_buf as a pooled heap buffer (f63c374), which replaces the earlier NodeFS::new_boxed() helper and shrinks every NodeFS::default() stack local with it.

CI on the previous head was green on 180 of 181 jobs. The one red job was test/js/web/url/url.test.ts on macOS x64 (an IDNA table test that depends on the runner's ICU version). It fails on main too and this diff does not touch it.

Reproduction (Windows x64, stock 1.4.1 and 1.4.1-canary):

let depth = 0;
const signal = AbortSignal.abort();
const body = {
  get [Symbol.asyncIterator]() {
    depth++;
    fetch("http://localhost/", { method: "POST", body, signal }).catch(() => {});
    return undefined;
  },
};
fetch("http://localhost/", { method: "POST", body, signal }).catch(() => {});
console.log(depth); // windows: 16, linux: 387

bun test test/js/web/fetch/body-async-iterator.test.ts with the stock canary on Windows: Expected: >= 32, Received: 16. With this branch (debug build) the same script prints 96 on Windows. The new test passed on every CI lane, including the Windows x64 and Windows aarch64 release builds.

The test cannot fail on Linux before the fix: PathBuffer is 4 KB there (387 levels before, 71 in a debug build after). The fail-before proof is Windows-only.

@coderabbitai

coderabbitai Bot commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: bbd269e0-7ab7-4276-830c-da0ce6af8fed

📥 Commits

Reviewing files that changed from the base of the PR and between 9781683 and daa2537.

📒 Files selected for processing (3)
  • src/runtime/node/node_fs.rs
  • src/runtime/webcore/fetch.rs
  • test/js/web/fetch/body-async-iterator.test.ts

Included review availability: Your plan provides up to 5 included reviews per hour; 0 remain after this review.


Walkthrough

Changes

Fetch filesystem safety

Layer / File(s) Summary
Pooled path storage and boxed NodeFS
src/runtime/node/node_fs.rs, src/runtime/webcore/fetch.rs
Filesystem URL resolution, Windows working-directory handling, file opening, and synchronous reads now use pooled path buffers. Synchronous reads construct boxed NodeFS instances.
Deep async-iterator regression coverage
test/js/web/fetch/body-async-iterator.test.ts
A subprocess test verifies that recursive aborted fetch() calls reach at least 32 levels without stderr output or process failure.

Suggested reviewers: dylan-conway, jarred-sumner

Merge Risk: ⚪ Minimal · up to daa25

The change moves fetch scratch buffers off the native stack while preserving fetch behavior, with targeted tests and probes covering the affected paths; no actionable merge-blocking risk remains beyond normal checks and review.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly identifies the main change: moving PathBuffer scratch storage to the pool for fetch_impl and NodeFS.
Description check ✅ Passed The description explains the problem, fix, rationale, regression test, verification results, and known pre-existing failures. It provides the information required by the repository template, although …
Full details: Description check

Explanation

The description explains the problem, fix, rationale, regression test, verification results, and known pre-existing failures. It provides the information required by the repository template, although it uses Problem and Fix headings instead of the exact template headings.


Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — mechanical swap to the documented pooled-buffer pattern.

What was reviewed:

  • PoolGuard<PathBuffer> derefs to the same PathBuffer type; all four call sites (decode_into, resolve_cwd_with_external_buf_z, join_abs_string_buf, getcwd, slice_z) still receive the same argument type via deref coercion, and every buffer is write-then-slice so stale pooled bytes are never read.
  • Verified the new_boxed SAFETY claim against the NodeFS struct: it is exactly PathBuffer (u8 array) + Option<NonNull<_>>, so all-zero is bit-identical to Default::default().
  • Pool re-entrancy: borrow_mut() is scoped to the pop(), and cache misses fall through to new_boxed, so taking 3 guards at once (Windows path) and recursing into fetch_impl while guards are live is fine.
  • Test uses a pre-aborted signal so no socket opens; threshold 32 sits between the unfixed Windows count (16) and the fixed debug counts (71 Linux / 96 Windows).
Extended reasoning...

Overview

The PR shrinks the native stack frame of fetch_impl by moving five PathBuffer-sized locals off the stack. Four PathBuffer::uninit() locals in src/runtime/webcore/fetch.rs become bun_paths::path_buffer_pool::get() guards, and the stack-local NodeFS::default() becomes a boxed NodeFS::new_boxed() added in src/runtime/node/node_fs.rs. A regression test in test/js/web/fetch/body-async-iterator.test.ts measures re-entrant fetch() depth via a Symbol.asyncIterator getter and asserts it reaches ≥ 32 (was 16 on Windows).

Security risks

None. No auth, crypto, permission, or input-validation logic is touched. The one new unsafe block (Box::<NodeFS>::new_zeroed().assume_init()) is sound: I checked the NodeFS struct definition — its only fields are a [u8; N]-backed PathBuffer and an Option<NonNull<VirtualMachine>>, both of which are valid at all-zero (the latter via the guaranteed null-pointer niche = None), matching the existing Default impl exactly. This is the same idiom already used in path_buffer_pool.rs itself. The test never opens a socket because the request carries a pre-aborted AbortSignal, so the recursion happens entirely in body extraction.

Level of scrutiny

Low-to-moderate. This is the textbook application of an in-tree pattern whose module doc (src/paths/path_buffer_pool.rs) names this exact problem, and src/CLAUDE.md prescribes it. PoolGuard<PathBuffer> implements Deref/DerefMut to PathBuffer, so every downstream consumer (resolve_cwd_with_external_buf_z takes &mut PathBuffer, getcwd takes &mut [u8], indexing/slicing) sees the identical type it saw before via coercion. The pool's RefCell::borrow_mut() is scoped to a single pop(), so holding three guards simultaneously and recursing into fetch_impl while they're live cannot deadlock or double-borrow; excess concurrent gets just heap-allocate. The buffers are not zeroed on reuse, but neither were the originals (PathBuffer::uninit()), and every use site writes-then-slices-by-returned-length. RAII via Drop on PoolGuard returns buffers on all early-return paths.

Other factors

No CODEOWNERS cover the changed files. No outstanding reviewer objections in the timeline. The test follows harness conventions (bunExe/bunEnv, await using, concurrent pipe drain, stderr/stdout asserted before exit code, appended to the existing module test file). The 32-level threshold is 2× the unfixed Windows number and well under the fixed debug numbers on both platforms, so it should not flake. The PR description is unusually thorough with concrete measurements, and the diff matches the claims.

@robobun

robobun commented Aug 28, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 12:09 AM PT - Aug 28th, 2026

✅ @robobun, your commit 74db05dad71e4fe678e6631efb700e34c92a5eb3 passed in Build #107422! 🎉


🧪   To try this PR locally:

bunx bun-pr 40680

That installs a local version of the PR into your bun-40680 executable, so you can run:

bun-40680 --bun

Comment thread src/runtime/node/node_fs.rs Outdated
Comment thread src/runtime/webcore/fetch.rs Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — mechanical swap to the documented pooled-buffer pattern, with a sound new_boxed() helper.

What was reviewed:

  • PoolGuard<PathBuffer> derefs to PathBuffer, so every call site (decode_into, resolve_cwd_with_external_buf_z, join_abs_string_buf, getcwd, slice_z) receives the same type as before; guards drop at scope exit on all early-return paths.
  • NodeFS::new_boxed() SAFETY: the struct is exactly [u8; N] + Option<NonNull<_>>, both valid at all-zeroes; mirrors the existing PoolStorage::new_boxed pattern in path_buffer_pool.rs.
  • Test: pre-aborted signal keeps recursion inside body extraction (no sockets), pipes drained concurrently, stderr/stdout asserted before exit code, threshold (32) sits between the unfixed Windows floor (16) and the fixed debug figure (96).
Extended reasoning...

Overview

The PR moves four stack-local PathBuffer::uninit() scratch buffers in fetch_impl (src/runtime/webcore/fetch.rs) onto the heap via bun_paths::path_buffer_pool::get(), and replaces a stack NodeFS::default() with a new NodeFS::new_boxed() helper in src/runtime/node/node_fs.rs. On Windows a PathBuffer is ~96 KB, so this shrinks the fetch_impl frame from ~297 KB to a size that lets recursive fetch() (via a body's Symbol.asyncIterator getter) go from 16 to ~96 levels before JSC's stack limit fires. A regression test in test/js/web/fetch/body-async-iterator.test.ts asserts at least 32 levels are reached.

Security risks

None. No new input parsing, no auth/crypto, no data exposure. The only unsafe addition is Box::<NodeFS>::new_zeroed().assume_init(); I checked the struct definition (line 4416-4424) — its two fields are a u8 array and Option<NonNull<VirtualMachine>>, both of which are valid when zero-filled (the latter via the null-pointer niche → None). This is the same pattern already used for PathBuffer/WPathBuffer in path_buffer_pool.rs:66.

Level of scrutiny

Low-to-moderate. This is the exact transformation src/CLAUDE.md prescribes ("prefer bun_paths::path_buffer_pool::get() over stack PathBuffer to avoid ~64 KB Windows stack frames"). PoolGuard implements Deref/DerefMut<Target = PathBuffer> and returns to the pool on Drop, so every borrow site type-checks against the same &mut PathBuffer/&mut [u8] as before, and every early return in the touched blocks releases the buffer via RAII. The getcwd call now passes &mut cwd_buf[..] explicitly, which is equivalent. The removed bun_paths::{self, PathBuffer} import is genuinely dead after the change.

Other factors

The test follows harness conventions: bunExe()/bunEnv, -e script, await using, Promise.all on stdout/stderr/exited, stderr and stdout asserted before exit code, no sleeps or hardcoded ports. The pre-aborted AbortSignal means no sockets open, so the test is hermetic. The bug hunt exited on dry_streak with zero findings. No CODEOWNERS entries cover the changed paths. Commits since the earlier COMMENTED review only shortened a comment and retriggered CI — no behavioral drift to re-examine. There are no outstanding CHANGES_REQUESTED reviews; the two github-actions inline threads were followed by the comment-shortening commit.

Comment thread src/runtime/webcore/fetch.rs Outdated
NodeFS embedded a PathBuffer by value, which made every NodeFS::default()
stack local 96 KB on Windows. The buffer now comes from
bun_paths::path_buffer_pool, so the struct is two words and the
NodeFS::new_boxed() workaround is not needed. The #[repr(C)] alignment
contract goes with it: the u16 reinterpretation relies on the heap
allocation's alignment, which the existing assert checks.
Comment thread src/runtime/node/node_fs.rs Outdated
@robobun robobun changed the title fetch: keep PathBuffer scratch out of the fetch_impl stack frame fetch, node:fs: take PathBuffer scratch from the pool in fetch_impl and NodeFS Aug 28, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

robobun added a commit that referenced this pull request Sep 5, 2026
…nc_error_buf

The pool stays in `bun_paths`, which `cargo miri test` covers, so its new
unit tests run under Miri. The one `bun_core` caller (`spawn_sync`, a cold
linux/freebsd path) uses a 4 KB `ZEROED` stack buffer instead.

`impl Default for PathBuffer` and `WPathBuffer` are gone. A zero-fill of a
98 KB buffer on Windows is now written as `ZEROED` at each site, and no
`#[derive(Default)]` can pick it up by accident. The crash handler uses
`ZEROED` explicitly.

`NodeFS::sync_error_buf` holds a pool guard. `NodeFS::default()` is a stack
local at 15 call sites, so a `ZEROED` field would have been a 98 KB memset
per call on Windows. This is the same change as #40680. `#[repr(C)]` goes
with it: the `u16` reinterpretation relies on the heap allocation's
alignment, which the existing `assert!` checks.
Jarred-Sumner pushed a commit that referenced this pull request Sep 6, 2026
…k arrays (#41442)

### Problem
- `PathBuffer::uninit()` and `WPathBuffer::uninit()`
(`src/bun_core/util.rs`) return a `[u8; N]` / `[u16; N]` built with
`MaybeUninit::uninit().assume_init()`. An integer must be initialized,
so this is undefined behavior at the constructor, before any byte is
read. Miri: `constructing invalid value of type bun_core::PathBuffer: at
.0[0], encountered uninitialized memory, but expected an integer`
(`cargo miri test -p bun_sys dir::tests::borrow_does_not_close`).
- `bufs_storage_init` (`src/resolver/resolver.rs`) does the same with
`Box::<Bufs>::new_uninit().assume_init()`. Both sites silenced the
`invalid_value` and `clippy::uninit_assumed_init` lints that catch this.

### Fix
- Remove both constructors and `impl Default` for both types. Every
scratch call site (about 440, mechanical one-line edits) takes a buffer
from `bun_paths::path_buffer_pool::get()` or its `w_` / `os_` siblings.
The pool zeroes a buffer once on allocation and reuses it, so there is
no per-call memset and no 98 KB stack frame on Windows. Structs built
per item (`NodeFS`, `bin::NamesIterator`, `PosixToWinNormalizer`) hold a
pool guard. Long-lived struct fields use an explicit `ZEROED`.
- The pool gains: a `try_with` fallback to the heap while TLS
destructors run, `Default` for `PoolGuard` so structs can derive it, and
`__lsan_ignore_object` on each allocation. The last one matters because
a guard live at `Global::exit` never drops, and in an optimized build
its stack slot is dead, so LeakSanitizer reported the buffer and the
ASAN lane aborted every `bun <file.md>` run.
- The resolver's `Bufs` uses `Box::new_zeroed()` (once per thread).
- Verified: `test/internal/source-lints/uninit-assumed-init.test.ts`
(its known-site list is now empty, so it fails on main), new pool unit
tests under Miri (`cargo miri test -p bun_paths`, 29 pass), a release
ASAN build with the CI `ASAN_OPTIONS` on the markdown, `bun exec` and
`bunx` paths, `cargo check` for linux, macOS, FreeBSD, Windows x64 and
arm64, clippy, and the fs, shell, spawn, path, resolve and `bun add`
suites.

### Background
- `PathBuffer` is a `[u8; MAX_PATH_BYTES]` newtype used as
write-then-read scratch around path syscalls. `MAX_PATH_BYTES` is 4 096
on Linux, 1 024 on macOS and 98 302 on Windows. `WPathBuffer` is the
`[u16; 32767]` sibling for Windows wide paths.
- "Every bit pattern is a valid `u8`" is true, but uninitialized memory
is not a bit pattern. The Rust reference lists an integer read from
uninitialized memory as undefined behavior.
- `bun_paths::path_buffer_pool` is a per-thread LIFO of up to four boxed
buffers with an RAII guard. It already existed with about 180 callers. A
cache miss allocates with `alloc_zeroed`, which is usually fresh
OS-zeroed pages.

<details><summary>Notes</summary>

Why not `ZEROED` at the call sites: on Windows that is a 98 KB memset
per call at about 440 sites. A previous attempt did exactly that and the
leak and stress tests timed out, which is why `uninit()` was introduced.
#31258 proposes `ZEROED` again and has the same cost.

Why not `[MaybeUninit<u8>; N]` inside `PathBuffer`: every syscall
wrapper in `bun_sys` takes `&mut [u8]`, so an initialized-prefix API
would change the whole syscall surface. The pool gives the same result
with no API change.

Why `impl Default` is gone: `Default` has to return a by-value buffer,
so it can only be `ZEROED`. With it, `#[derive(Default)]` on a struct
that embeds a `PathBuffer` silently picked up a 98 KB memset per
construction on Windows (`PosixToWinNormalizer`, built per resolved
import). Every zero-fill is now a visible `ZEROED` at its site: the
crash handler (must not allocate), `bun_core::spawn_sync` (cold,
linux/freebsd), and once-per-task fields in `PackageInstaller`,
`extract_tarball`, the test `Scanner`, `Tree` iterators, `PatchFile`,
`WindowsWatcher` and the resolver's `top_level_dir_buf`.

Overlap with #40680: that PR moves `NodeFS::sync_error_buf` to a pool
guard and the `fetch_impl` locals to the pool for Windows stack depth.
This PR contains the same source changes (the `fetch_impl` locals were
part of the sweep). #40680's test is not included here.

LeakSanitizer and `Global::exit`: reproduced with `bun run build:asan`
and `ASAN_OPTIONS=detect_leaks=1:abort_on_error=1`.
`render_markdown_file_and_exit` held two guards at `exit`, and LSan
reported the 4 KB buffer allocated at `path_buffer_pool::get`
(`run_command.rs:3423`). `new_boxed` now calls `__lsan_ignore_object` on
each buffer (under `cfg(bun_asan)`), which fixes every such site. The
markdown body was also split into a function that returns the exit code.
Verified both ways: with the original `-> !` shape plus only the LSan
call, the test passes.

Perf check (debug builds, main vs this branch, three runs each):
`Glob("**/*.rs").scan` over `src/` 457/475/459 ms vs 478/453/452 ms, 20
000 `statSync` + `realpathSync` 2663/2674/2663 ms vs 2663/2717/2683 ms,
2 000 `readdirSync` 617/618/617 vs 617/628/626 ms, 4 000
`Bun.resolveSync` 261/262/262 vs 263/266/265 ms. No difference outside
noise. No Windows measurement. Windows has the most to gain (no 98 KB
stack frames) and nothing in the hot paths zero-fills per call.

`bun_sys` cannot join `MIRI_CRATES` in `scripts/rust-miri.ts`: after
this change Miri gets past the constructor and stops at `can't call
foreign function openat64`, which it has no shim for. The pool and its
unit tests stay in `bun_paths`, which is in `MIRI_CRATES`.

`test/js/bun/glob/scan.test.ts` times out in this container on main and
on this branch alike: it scans the whole checkout, which holds 685 000
files of build output here.

Test suites run with the debug build: `test/js/node/fs/fs.test.ts` (565
pass), `test/js/node/fs/fs-mkdir.test.ts`,
`test/js/bun/shell/bunshell.test.ts` (431 pass),
`test/js/bun/spawn/spawn.test.ts` (148 tests),
`test/js/node/path/path.test.js`, `test/js/bun/resolve/` (390 pass, one
5 s timeout that also fails on main here),
`test/cli/install/bun-add.test.ts` (71 pass),
`test/cli/run/markdown-entrypoint.test.ts`. With the release ASAN build
and the CI leak settings: `test/cli/run/` (1177 pass, 4 environment
failures: FUSE, uid/gid, a 60 s timeout) and
`test/js/bun/shell/exec.test.ts`.
</details>

<!-- robobun:evidence:begin -->

---

**[human-review]** gate passed · iteration 1 · 129 files touched

<details><summary>fails on main (without fix)</summary>

```console
ASAN without fix: 1 FAILED
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" test/internal/source-lints/uninit-assumed-init.test.ts
bun test v1.4.3 (e0a2b82)

test/internal/source-lints/uninit-assumed-init.test.ts:
(pass) scans a non-empty set of tracked Rust sources [1.99ms]
68 | });
69 | 
70 | test("no new #[allow(invalid_value)] or #[allow(clippy::uninit_assumed_init)]", () => {
71 |   // `[^)]*` spans the multi-line form of the attribute; lint paths never
72 |   // contain `)`.
73 |   expect(scan(/#!?\[(?:allow|expect)\([^)]*\b(?:invalid_value|clippy::uninit_assumed_init)\b/)).toEqual(KNOWN_SITES);
                                                                                                     ^
error: expect(received).toEqual(expected)

- []
+ [
+   "src/bun_core/util.rs",
+ ]

- Expected  - 1
+ Received  + 3

      at <anonymous> (/workspace/bun/test/internal/source-lints/uninit-assumed-init.test.ts:73:97)
(fail) no new #[allow(invalid_value)] or #[allow(clippy::uninit_assumed_init)] [126.91ms]

 1 pass
 1 fail
 2 expect() calls
Ran 2 tests across 1 file. [23.59s]
error: script "bd" exited with code 1
__F:1:S:0

release without fix: 1 FAILED
bun test v1.4.3-canary.1 (1491acd)

test/internal/source-lints/uninit-assumed-init.test.ts:
(pass) scans a non-empty set of tracked Rust sources [0.11ms]
68 | });
69 | 
70 | test("no new #[allow(invalid_value)] or #[allow(clippy::uninit_assumed_init)]", () => {
71 |   // `[^)]*` spans the multi-line form of the attribute; lint paths never
72 |   // contain `)`.
73 |   expect(scan(/#!?\[(?:allow|expect)\([^)]*\b(?:invalid_value|clippy::uninit_assumed_init)\b/)).toEqual(KNOWN_SITES);
                                                                                                     ^
error: expect(received).toEqual(expected)

- []
+ [
+   "src/bun_core/util.rs",
+ ]

- Expected  - 1
+ Received  + 3

      at <anonymous> (/workspace/bun/test/internal/source-lints/uninit-assumed-init.test.ts:73:97)
(fail) no new #[allow(invalid_value)] or #[allow(clippy::uninit_assumed_init)] [12.05ms]

 1 pass
 1 fail
 2 expect() calls
Ran 2 tests across 1 file. [561.00ms]
__F:1:S:0
```

</details>

<details><summary>passes on PR (with fix)</summary>

```console
ASAN with fix: all passed
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" test/internal/source-lints/uninit-assumed-init.test.ts
bun test v1.4.3 (e0a2b82)

test/internal/source-lints/uninit-assumed-init.test.ts:
(pass) scans a non-empty set of tracked Rust sources [2.21ms]
(pass) no new #[allow(invalid_value)] or #[allow(clippy::uninit_assumed_init)] [122.33ms]

 2 pass
 0 fail
 2 expect() calls
Ran 2 tests across 1 file. [24.05s]
__F:0:S:0

release with fix: all passed
$ bun scripts/build.ts --profile=release
[configured] bun-profile → bun (stripped) in 575ms (unchanged)
ninja: Entering directory `/workspace/bun/build/release'
[1/5] gen generated_host_exports.rs
generated_host_exports.rs: 122 exports (host=5, lazy=10, generic=107, rust=0); 242 extern-C blocks audited
[1/5] cargo bun_runtime → libbun_runtime.a
�[1m�[92m   Compiling�[0m bun_brotli_sys v0.0.0 (/workspace/bun/src/brotli_sys)
�[1m�[92m   Compiling�[0m bun_core v0.0.0 (/workspace/bun/src/bun_core)
�[1m�[92m   Compiling�[0m bun_errno v0.0.0 (/workspace/bun/src/errno)
�[1m�[92m   Compiling�[0m bun_ptr v0.0.0 (/workspace/bun/src/ptr)
�[1m�[92m   Compiling�[0m bun_boringssl_sys v0.0.0 (/workspace/bun/src/boringssl_sys)
�[1m�[92m   Compiling�[0m bun_safety v0.0.0 (/workspace/bun/src/safety)
�[1m�[92m   Compiling�[0m bun_base64 v0.0.0 (/workspace/bun/src/base64)
�[1m�[92m   Compiling�[0m bun_cares_sys v0.0.0 (/workspace/bun/src/cares_sys)
�[1m�[92m   Compiling�[0m bun_zlib_sys v0.0.0 (/workspace/bun/src/zlib_sys)
�[1m�[92m   Compiling�[0m bun_zstd v0.0.0 (/workspace/bun/src/zstd)
�[1m�[92m   Compiling�[0m bun_picohttp v0.0.0 (/workspace/bun/src/picohttp)
�[1m�[92m   Compi
... (truncated)
```

</details>

<details><summary>diff hotspot</summary>

```
src/bun_core/lib.rs                                |   9 ++
 src/bun_core/util.rs                               |  49 ++-------
 src/bundler/OutputFile.rs                          |   7 +-
 src/bundler/linker_context/computeChunks.rs        |   4 +-
 .../linker_context/generateChunksInParallel.rs     |   2 +-
 .../linker_context/writeOutputFilesToDisk.rs       |   6 +-
 src/bundler/options.rs                             |   4 +-
 src/bundler/transpiler.rs                          |   4 +-
 src/bunfig/arguments.rs                            |   6 +-
 src/crash_handler/lib.rs                           |   4 +-
 src/dotenv/env_loader.rs                           |   4 +-
 src/event_loop/MiniEventLoop.rs                    |   2 +-
 src/glob/GlobWalker.rs                             |  14 ++-
 src/install/PackageInstall.rs                      |  12 +--
 src/install/PackageInstaller.rs                    |  22 ++--
 src/install/PackageManager.rs                      |  14 +--
 src/install/PackageManager/CommandLineArguments.rs |   6 +-
 .../PackageManager/PackageManagerDirectories.rs    |  16 +--
 .../PackageManager/PackageManagerEnqueue.rs        |   8 +-
 .../PackageManager/PackageManagerOptions.rs        |   9 +-
 .../PackageManager/PackageManagerResolution.rs     |   3 +-
 .../PackageManager/WorkspacePackageJSONCache.rs    |   4 +-
 src/install/PackageManager/patchPackage.rs         |  22 ++--
 .../PackageManager/updatePackageJSONAndInstall.rs  |   9 +-
 src/install/TarballStream.rs                       |   6 +-
 src/install/bin.rs                                 |  12 +--
 src/install/extract_tarball.rs                     |  14 ++-
 src/install/hoisted_install.rs                     |   4 +-
 src/install/isolated_install.rs                    |   6 +-
 src/install/lib.rs                                 |   6 +-
 src/install/lockfile.rs                            |   8 +-
 src/install/lockfile/Package.rs                    |   8 +-
 src/install/loc
... (truncated)
```

</details>

**gate history** · 2 passed · 0 rejected · iteration 1

<details><summary>evidence per changed file</summary>

```
file                                                    reads  edits  tests
src/bun_core/lib.rs                                         1      2     11
src/bun_core/util.rs                                        1      3     14
src/bundler/OutputFile.rs                                   0      0      9
src/bundler/linker_context/computeChunks.rs                 0      0      9
src/bundler/linker_context/generateChunksInParallel.rs      0      0      9
src/bundler/linker_context/writeOutputFilesToDisk.rs        0      0      9
src/bundler/options.rs                                      0      0      9
src/bundler/transpiler.rs                                   0      0      9
src/bunfig/arguments.rs                                     0      0      9
src/crash_handler/lib.rs                                    0      0      9
src/dotenv/env_loader.rs                                    0      0      9
src/event_loop/MiniEventLoop.rs                             0      0      9
src/glob/GlobWalker.rs                                      0      0     11
src/install/PackageInstall.rs                               0      0      9
src/install/PackageInstaller.rs                             0      0      9
src/install/PackageManager.rs                               0      0      9
(+ 113 more files)
```

</details>

**root cause** · written by the author bot

`PathBuffer::uninit`, `WPathBuffer::uninit`, and the resolver's
`bufs_storage_init` materialized `[u8; N]` and `[u16; N]` values from
`MaybeUninit::uninit().assume_init()`, which is undefined behavior
because integer types require initialized memory regardless of whether
the bytes are later read, and the `invalid_value` and
`uninit_assumed_init` lints that diagnose this were suppressed at those
sites. The fix removes those constructors and migrates their call sites
to the existing `path_buffer_pool`, which produces buffers through
`Box::new_zeroed().assume_init()` so every byte is written b…

<!-- robobun:evidence:end -->
@robobun

robobun commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator Author

Closing: the source changes here landed on main in #41442 (e854103), which moved every PathBuffer scratch to the pool including the fetch_impl locals and NodeFS::sync_error_buf.

@robobun

robobun commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator Author

The test from this PR is now proposed on its own in #41493.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants