Skip to content

FormData: read a Bun.file() part whose stat size is 0 - #41590

Open
robobun wants to merge 1 commit into
mainfrom
robobun/5049e0bb/st-size-hint-readers
Open

robobun wants to merge 1 commit into
mainfrom
robobun/5049e0bb/st-size-hint-readers

Conversation

@robobun

@robobun robobun commented Sep 6, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • FormData.append("f", Bun.file("/proc/version")) serializes an empty part, with no error. A procfs file is a regular file that reports st_size 0 and has content. Bun.file(p).text() returns the content.
  • Cause: the multipart serializer calls resolve_size() before the read (Blob.rs:3666). That stores the stat size 0 as the size of the blob. The read then gets max_size = Some(0) (Blob.rs:3689) and reads nothing.

Fix

  • The serializer no longer calls resolve_size(). A whole-file part reaches NodeFS::read_file with the "size unknown" sentinel, as a fetch body does.
  • read_file is unchanged. A device, a FIFO and a file with no end stay bounded at one scratch buffer.
  • Verified: test/js/web/html/FormData.test.ts, three new tests. The procfs test fails on bun 1.4.3.

Background

Downsides

Notes

History of this PR. The first version changed three sites: the FormData serializer, NodeFS::read_file and the module loader. Two problems showed up after the push.

  1. fetch and FormData with Bun.file("/dev/zero") read without bound (RSS passed 3 GB in 3 s on a debug build). A condition on S_ISREG fixed that.
  2. A review of that version found that a regular file can also have no practical end. /proc/self/pagemap is S_ISREG with st_size 0 and yields hundreds of GB. With the read_file and loader changes, an upload, a FormData part or a text import of it passed 1.5 GB RSS in 2 to 4 s. bun 1.4.3 returns at once. The same review found that .slice(1) and .slice(0, 1 << 20) of a procfs file were still cut at 262160 bytes.

So this PR now holds only the serializer change, which adds no read past the bound that main has. The other two sites need a decided bound. They are #43964 (NodeFS::read_file) and #43965 (module loader).

Bytes inside the part (bun 1.4.3 canary 367d939, and a debug build of this branch):

part bun 1.4.3 this PR
/proc/version (219 B) 0 219
/proc/kallsyms (12 MB) 0 262160
/proc/self/pagemap 0 262160, returns at once
/dev/zero 262160 262160
ordinary file, 400000 B 400000 400000

Syscall counts. Counted with a ptrace counter on debug builds of main (4227e46) and of this branch, because strace is not installed in the container. Only calls on the part file count.

part main this PR
1000 B openat, newfstatat, fstat, read, read(len 0), close openat, read, read, close
400000 B openat, newfstatat, fstat, read x2, read(len 0), close openat, fstat, read x3, close

The newfstatat that goes away is the stat in resolve_size().

The 16 bytes. read_file sizes its buffer as the fstat size plus 16 (node_fs.rs, initial_cap). With a caller bound it does not grow past that. On main the bound was the size that resolve_size() stored, so the part ended exactly there.

Related open PRs. #43910 (draft) stops resolve_size() from storing a stat size on a file blob. The two changes are independent and merge in either order. #41488 streams a device or FIFO body of fetch. #41593 is the body stream site.

Not covered here. --env-file of a procfs file loads nothing (src/dotenv/env_loader.rs, read_env_file_contents). Node loads it. That is a separate report.

Suites run on the debug build. test/js/web/html/FormData.test.ts (152 pass), and on the earlier three-site version blob.test.ts, bun-file-read.test.ts, fetch-file-upload.test.ts, node/fs/fs.test.ts.

Self-review. Three rounds. Round 1 moved the fix from the call sites to read_file. Round 2 (another session) found the /dev/zero read. Round 3 found the /proc/self/pagemap read, the slices, and asked for one site per PR. This version is the result.


no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/web/html/FormData.test.ts, test/js/bun/util/text-loader.test.ts, test/js/bun/http/fetch-file-upload.test.ts

@coderabbitai

coderabbitai Bot commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: oven-sh/bun/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 3d92c352-4e02-4f5c-959b-3ec45b64ecd0

📥 Commits

Reviewing files that changed from the base of the PR and between ad49337 and a92c100.

📒 Files selected for processing (4)
  • src/resolver/fs.rs
  • src/runtime/node/node_fs.rs
  • src/runtime/webcore/Blob.rs
  • test/js/bun/util/text-loader.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 4 remain after this review.


Walkthrough

Resolver readers now read regular files to EOF when the reported size is smaller than the probe. NodeFS treats BLOB_SIZE_MAX as unbounded for regular files. FormData file reads pass the current size through, with tests for procfs files, sliced files, and device streams.

Changes

File read size handling

Layer / File(s) Summary
Read regular files to EOF
src/resolver/fs.rs, test/js/bun/util/text-loader.test.ts
Resolver readers check whether an undersized reported size belongs to a regular file. They then read to EOF. A Linux procfs test checks that text import matches the file contents.
Runtime file reads and upload coverage
src/runtime/node/node_fs.rs, src/runtime/webcore/Blob.rs, test/js/web/html/FormData.test.ts, test/js/bun/http/fetch-file-upload.test.ts
FormData file entries pass the current size as the read limit. NodeFS treats BLOB_SIZE_MAX as unbounded for regular files and retains bounds for non-regular files. Tests cover procfs files, sliced files, and device streams.

Suggested reviewers: jarred-sumner

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to a92c1

The change addresses the reported procfs reads while retaining bounds for device streams. No issue identified here needs to block merging after normal checks.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly identifies the main change: reading a FormData Bun.file() part when its reported stat size is zero.
Description check ✅ Passed The description explains the problem, root cause, fix, verification, behavior for related file types, limitations, and test results. It does not use the exact template headings, but it covers the requ…

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Sep 6, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: reproduced on bun 1.4.3. A FormData part for Bun.file("/proc/version") is empty, and Bun.file("/proc/version").text() has the content. The procfs test in test/js/web/html/FormData.test.ts fails on 1.4.3 and passes with this branch (73cc514).

This PR now holds only the FormData serializer change. Earlier pushes also changed NodeFS::read_file and the module loader. With those changes, an upload, a FormData part or a text import of /proc/self/pagemap read without bound (over 1.5 GB RSS in 2 to 4 s), and the first push did the same for /dev/zero. bun 1.4.3 returns at once in both cases, and so does this branch. The two other sites go to separate PRs with a decided bound.

@github-actions github-actions Bot added the claude label Sep 6, 2026
@robobun

robobun commented Sep 6, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 3:06 AM PT - Sep 25th, 2026

❌ @robobun, your commit 73cc514 has 1 failures in Build #120573 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 41590

That installs a local version of the PR into your bun-41590 executable, so you can run:

bun-41590 --bun

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Additional findings (outside the current diff — GitHub can't attach inline comments there):

  • 🟡 src/runtime/webcore/Blob.rs — nit: The doc comment on the local FormDataEntry mirror says on_entry "needs &mut Blob to call resolve_size()", but this PR removes the resolve_size() call from on_entry (and resolve_size takes &self anyway). The stated justification for driving DOMFormData__forEach with a mutable-blob variant instead of the stock bun_jsc::dom_form_data::FormDataEntry is now dead. Fix: update or delete the comment; if nothing else in on_entry needs &mut Blob, drop the local mirror and use the immutable FormDataEntry per REVIEW.md's "delete dead code in the same PR that makes it dead".

    Extended reasoning...

    The removed hunk at Blob.rs:3763-3766 was the only reason cited (Blob.rs:3574-3576) for the FormDataEntry<'a>::File { blob: &'a mut Blob } local enum and its custom DOMFormData__forEach driver. After the removal, on_entry reads only blob.content_type_slice(), blob.store.get(), blob.offset.get() and blob.size.get() — all &self/Cell accessors. The comment now points a future reader at a call that no longer exists, and the &mut variant it justifies is left as unexplained duplication of bun_jsc::dom_form_data::FormDataEntry. No runtime failure; this is documentation/dead-code debt this PR introduces (the comment's reasoning was already shaky pre-PR since resolve_size is fn(&self) at Blob.rs:2188, but the referenced call did exist on the base branch).

    Verification: nit — The doc comment at src/runtime/webcore/Blob.rs:3573-3576 reads "FormDataContext::on_entry needs &mut Blob to call resolve_size(), so we drive the C++ DOMFormData__forEach directly with this mutable variant." This PR's only change to Blob.rs is deleting the blob.resolve_size() call at (former) lines 3764-3766 from on_entry; the comment now cites a call that no longer exists.…

Comment thread test/js/bun/http/fetch-file-upload.test.ts Outdated
Comment thread test/js/bun/util/text-loader.test.ts Outdated
Comment thread src/resolver/fs.rs Outdated
Comment thread src/resolver/fs.rs
Comment thread src/runtime/node/node_fs.rs Outdated
Comment thread src/runtime/webcore/Blob.rs Outdated
@robobun
robobun force-pushed the robobun/5049e0bb/st-size-hint-readers branch from ad49337 to a92c100 Compare September 24, 2026 23:43

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Comment thread src/resolver/fs.rs Outdated
Comment thread src/runtime/node/node_fs.rs Outdated
A procfs file is a regular file that reports st_size 0 and has
content. The multipart serializer called resolve_size() before the
read. That stored the stat size 0 as the size of the blob, so the read
got max_size = Some(0) and the part was empty.

The serializer no longer resolves the size. A whole-file part reaches
NodeFS::read_file with the "size unknown" sentinel, as a fetch body
does. read_file is unchanged: a part is still bounded by its scratch
buffer when the stat size is wrong, and a device or FIFO is read as
before.

The FormDataEntry doc comment named the removed call and a type that
does not exist.
@robobun
robobun force-pushed the robobun/5049e0bb/st-size-hint-readers branch from a92c100 to 73cc514 Compare September 25, 2026 09:41
@robobun robobun changed the title Read procfs files to EOF in fetch uploads, FormData parts and the text loader FormData: read a Bun.file() part whose stat size is 0 Sep 25, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline finding, I also checked the new /dev/zero child test for a hang: Some(MAX_SIZE) is still is_some() at src/runtime/node/node_fs.rs:7211, so both the pre-stat scratch read and the post-fstat loop stay capped, and the sliced-part test is unaffected because get_slice_from stores an explicit offset/len that never went through resolve_size(). One scope note: the diff at HEAD contains only the Blob.rs hunk and the FormData test, so the node_fs.rs, resolver/fs.rs and fetch-upload changes the description and title cover are not in this push.

Extended reasoning...

The push removes the resolve_size() call before the FormData file-part read in src/runtime/webcore/Blob.rs and adds three FormData tests; no security-sensitive surface is touched. The scope of the diff is much narrower than the PR title and description, which still describe changes to read_file, the resolver loader and fetch uploads that are absent here, and the inline finding shows the FormData fix only covers procfs files up to the 256 KiB scratch buffer.

Still open from earlier reviews (1):

  • Unresolved: 1 minor or pre-existing.

Additional findings (outside the current diff — GitHub can't attach inline comments there):

  • 🟣 src/runtime/webcore/Blob.rs — A FormData part (or fetch body) for a procfs file over 256 KiB is still cut at 262160 bytes after this change; the PR text says otherwise. Blob.rs:3683 now passes the unresolved MAX_SIZE sentinel as max_size, and node_fs.rs:7211 has_max_size = args.max_size.is_some() treats any Some as a real bound, so the loop at node_fs.rs:7282 never grows past the scratch read. The described read_file change ("reads to EOF for the whole-file sentinel when the file is regular") is not in this diff, nor are the fetch/text-loader tests. …

    Why this was flagged

    …Fix: in read_file treat a whole-file sentinel bound as "no bound" for regular files (covering both Blob.rs:3683 and fetch.rs:1655), and add a >256 KiB procfs case to the test; /proc/version is consumed entirely by the pre-stat read at node_fs.rs:7121 and cannot detect this.

    formData.append("f", Bun.file("/proc/kallsyms")) then new Response(formData).bytes(), or fetch(url, { body: Bun.file("/proc/kallsyms") }), on a host where that file exceeds 256 KiB. With resolve_size() gone, Blob.rs:3683 sets rf_args.max_size = Some(blob.size.get()) where the size is still MAX_SIZE; fetch.rs:1655 does the same on the base and on this branch. In node_fs.rs:7211 has_max_size = args.max_size.is_some() is true for that sentinel, so after the 256 KiB pre-stat read (node_fs.rs:7115-7131) the post-fstat loop reads only the 16 spare bytes of initial_cap (node_fs.rs:7233-7237) and the growth arm at node_fs.rs:7282 is skipped because !has_max_size is false; the next read gets an empty slice, returns 0, and the part is 262160 bytes. The base branch serialized 0 bytes for this…

    Verification: normal — triggered whenever an unsliced Bun.file() FormData part refers to a regular file whose st_size is 0 (procfs) but whose content exceeds 256 KiB, e.g. /proc/kallsyms. Mechanism verified: git diff --stat shows only src/runtime/webcore/Blob.rs and test/js/web/html/FormData.test.ts changed — the read_file "regular file reads to EOF for the whole-file sentinel" change, the fetch change…

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants