Skip to content

blob: report the size of Blobs created by the native bindings to the GC - #37697

Open
robobun wants to merge 4 commits into
mainfrom
farm/6e8db77d/blob-bindings-estimated-size
Open

robobun wants to merge 4 commits into
mainfrom
farm/6e8db77d/blob-bindings-estimated-size

Conversation

@robobun

@robobun robobun commented Aug 12, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • Blobs that bun builds natively tell the GC they are about 48 bytes whatever their payload: a File parsed from a multipart Request or Response body (and the FormData holding it, 81), a WebSocket message received with binaryType = "blob" (and its MessageEvent, 56), and Bun.WebView#screenshot(). new Blob() of the same 1 MiB reports 1048896. Reproduces on 1.4.0 and main.
  • Cause: a Blob's estimated size is a cached field that only the JS Blob constructor and the ordinary Rust to JS conversion fill in. The native entry points hand the Blob to JSC with the cache still 0, and duplicating a Blob copies its source's 0.
  • The bytes are still freed by the periodic event loop GC. What is wrong is the number JSC uses to schedule collections, and what heap snapshots and estimateShallowMemoryUsageOf show.

Fix

  • The four native entry points (dupe, from bytes, from bytes with type, from mmap with type) and the Blob constructor go through one helper that computes the estimate and then moves the Blob to the heap. The typed variants set the content type before the size is taken, so it is counted.
  • Correct because at these points the Blob is freshly built, exclusively owned, on the JS thread, and has its final fields, and every later reader (the JS wrapper, FormData, MessageEvent) reads that same allocation. The value only goes to JSC, and it now matches new Blob(sameBytes) plus the stored name and content type.
  • Producers with the same symptom (new File(...), Bun.Image#blob(), server and bundler accounting) are left to Report File and S3File sizes to the GC when their wrappers are created #37667, Blob: wrap blobs for JS through JsClass::to_js only #37656 and a separate issue. Moving the computation into Blob::new itself is a follow-up once Blob: wrap blobs for JS through JsClass::to_js only #37656 lands.
  • Verification: new heap-snapshot tests for the multipart, re-appended File and WebSocket cases fail on the released bun and pass with the fix. One assertion was added to each existing screenshot test; the Chrome one was run locally against a real Chrome (48 before, 1123 after, for a 370 byte PNG), the WebKit one is left to CI.

Background

  • JSC only sees memory it allocated itself. A native object holding a buffer outside the JS heap reports that size as extra memory, once at creation and again at every GC, so JSC knows when a collection is worth running. The number is advisory and does not decide when the object is freed.
  • Blob's estimate is a cache because the accessor is also called from the GC marking thread, so it never computes anything. Whoever builds the Blob has to compute it once, on the JS thread, before handing it over.
  • A Blob is assembled on the stack and then Blob::new moves it to a heap allocation whose pointer the C++ bindings wrap in a JS object. The cache has to be filled before that move and after every field it counts (name, content type) is set.
  • C++ code (multipart parsing, WebSocket, WebView) creates Blobs through a handful of extern "C" exports rather than the JS constructor, so a fix in the constructor never reaches them. Dupe copies an existing Blob, cache included.
  • estimateShallowMemoryUsageOf from bun:jsc returns the size JSC currently holds for one object, which is how the tests observe the number.

no test proof · iteration 0 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/bun/webview/webview-chrome.test.ts test/js/bun/webview/webview.test.ts

Original description

Repro

import { estimateShallowMemoryUsageOf } from "bun:jsc";

const payload = Buffer.alloc(1 << 20, "x");
console.log(estimateShallowMemoryUsageOf(new Blob([payload]))); // 1048896

const upload = new FormData();
upload.append("f", new Blob([payload]), "u.bin");
const parsed = await new Response(upload).formData();
console.log(parsed.get("f").size);                         // 1048576
console.log(estimateShallowMemoryUsageOf(parsed.get("f"))); // 48  (the bare cell)
console.log(estimateShallowMemoryUsageOf(parsed));          // 81

// Same for a WebSocket with ws.binaryType = "blob": a 1 MiB binary message
// arrives as a Blob whose estimate is 48, inside a MessageEvent whose estimate is 56.
// Same for Bun.WebView#screenshot(): a 370 byte PNG Blob reports 48.

Reproduces on bun 1.4.0 and on main (Request#formData() behaves the same as Response#formData()).

Cause

Blob.reported_estimated_size is a cache. The generated Blob__estimatedSize (what Blob__create's reportExtraMemoryAllocated, JSBlob::visitChildren, JSBlob::memoryCost and therefore WebCore::Blob::memoryCost() read) only returns it, because it is also called from the GC marking thread. Whoever creates a Blob that JSC will account for has to fill it first; BlobExt::to_js and the Blob constructor do.

The Blobs above never pass through either of those. They are created by the exports behind the C++ WebCore::Blob wrapper (src/jsc/bindings/blob.h) and the webview backends, and none of them computed the size:

  • Blob__dupe: every WebCore::Blob to JS conversion (src/jsc/bindings/blob.cpp: Blob__setAsFile, then Blob__dupe, then Blob__create), every multipart file entry (FormData.rs builds a stack Blob and appendBlob dupes it) and every formData.append(name, blob) from JS (Blob__dupeFromJS). dupe_with_content_type copies the source's cached value, which for a multipart entry is the never-computed 0, so the 0 was copied into the FormData entry and again into the JS wrapper. Blob__create then reported 0 bytes at allocation and the marking thread reported 0 at every GC.
  • Blob__fromBytes: WebSocket binary messages in blob mode (WebSocket.cpp). The JS-visible Blob is a Blob__dupe of this one, and MessageEvent::memoryCost() reads this one directly.
  • Blob__fromBytesWithType / Blob__fromMmapWithType: Bun.WebView#screenshot() on the Chrome and WebKit backends, which hand the pointer straight to Blob__create.

Fix

The four exports, and the Blob constructor (which already did the same two steps inline), go through one helper, new_for_bindings, which calls calculate_estimated_byte_size() and then Blob::new. Blob__fromBytes / Blob__fromBytesWithType / Blob__fromMmapWithType are restructured so the content type is set on the stack Blob before the size is taken; the two copies of the content type stamping become set_content_type_from_cstr. The 'static requirement on mime in the two *WithType doc comments is dropped, since the string has been copied into an owned content_type since #29910.

Why this is the right place: these exports are exactly the points where a Blob is handed to a consumer that reads the cache without going through BlobExt::to_js, and the freshly built Blob is exclusively owned and on the JS thread there, so computing is safe and covers every consumer of that allocation at once (the JS wrapper, DOMFormData::memoryCost, MessageEvent::memoryCost). Computing in Blob__dupe rather than at the toJS call site also means the estimate includes the file name that Blob__setAsFile stores just before it, and it fixes the JS append() path for a source whose own cache was never filled (a parsed entry appended to another FormData). The computation is a handful of field reads, negligible next to the heap allocation and wrapper creation in the same call. Behaviour is otherwise unchanged: the value is only reported to JSC, and it now reports what new Blob(sameBytes) reports (plus the stored name / content type). This does not produce unbounded retention on its own, since Bun's periodic event loop GC still collects these Blobs eventually; what was wrong is the accounting JSC uses to schedule collections and what heap snapshots and estimateShallowMemoryUsageOf show.

Sibling paths with the same symptom, intentionally not in this PR:

Where this should end up: Blob::new is the one heap-promotion point every reader of the cache goes through, so once #37656 has moved calculate_estimated_byte_size into bun_jsc (it has to live next to Blob::new to be called from there), the computation belongs in Blob::new itself, and new_for_bindings, the constructor's call and the explicit calls added by #37656 / #37667 all become deletable. Every current Blob::new caller either has its fields final at that point or recomputes in BlobExt::to_js afterwards, so that move is mechanical. It is not done here because it needs the hoist that #37656 is already making; this PR keeps the fix in bun_runtime so the three PRs stay independent, and the consolidation is a follow-up on top of whichever lands last.

Verification

Four tests added at the end of the "Native types report their size correctly" group in test/js/bun/util/heap-snapshot.test.ts, each asserting the estimate is between 1x and 2x a 1 MiB payload:

  • File parsed from a multipart Request body, and from a multipart Response body: the File itself and the parsed FormData (the latter goes through the entry's WebCore::Blob, i.e. the appendBlob dupe). Without the fix: 48 and 81.
  • A parsed multipart File appended to another FormData (Blob__dupeFromJS with an uncomputed source). Without the fix: 84.
  • A WebSocket message received with binaryType = "blob": event.data (the Blob__dupe) and the MessageEvent itself (MessageEvent::memoryCost() reads the Blob__fromBytes Blob, so this assertion is the one that depends on Blob__fromBytes computing). Without the fix: 48 and 56.

The two *WithType exports are covered by one added assertion in each existing screenshot test: test/js/bun/webview/webview-chrome.test.ts ("screenshot returns a PNG Blob", Blob__fromBytesWithType, runs on the Linux lanes that have Chrome) and test/js/bun/webview/webview.test.ts (same name, Blob__fromMmapWithType, macOS lanes). Each asserts the screenshot Blob reports at least what new Blob([itsBytes]) reports. The Chrome one was run here through a real Chrome: 48 vs 690 for a 370 byte PNG without the fix, 1123 vs 1114 with it (the 9 extra bytes are the owned image/png); the WebKit one is the same code path on a Bytes store and is left to CI. The existing Blob constructor coverage (blob.test.ts, the FormData test in the same group) still passes with the constructor routed through the helper.

With the fix the debug build reports 1049325 for the parsed File (new Blob of the same bytes: 1049320, the difference being the stored u.bin name), 1049361 for the FormData, 1049320 for the WebSocket Blob and 1049336 for its MessageEvent.

The heap-snapshot tests fail with USE_SYSTEM_BUN=1 bun test and pass with bun bd test. Also run on the fixed debug build: test/js/web/html/FormData.test.ts, FormData-multipart-serialization.test.ts, FormData-file-error-leak.test.ts, test/js/web/fetch/blob.test.ts, test/js/web/websocket/websocket-blob.test.ts (all pass; blob-file-name-ownership.test.ts and two pre-existing tests in heap-snapshot.test.ts only exceed the 5s default per-test timeout locally under debug+ASAN, which they also do without this change). cargo fmt --check is clean.

Blob.reported_estimated_size is a cache that Blob__estimatedSize only reads
(it also runs on the GC thread). BlobExt::to_js and the Blob constructor fill
it, but the exports the C++ WebCore::Blob wrapper uses to create Blobs did
not, so multipart formData() entries, WebSocket binaryType = "blob" messages
and the FormData / MessageEvent objects holding them reported 0 extra bytes.

Blob__dupe, Blob__fromBytes, Blob__fromBytesWithType and Blob__fromMmapWithType
now compute the size before heap-promoting the Blob they hand back.
@coderabbitai

coderabbitai Bot commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Changes

Blob memory accounting

Layer / File(s) Summary
Binding-aware Blob construction
src/runtime/webcore/Blob.rs
Blob constructors estimate memory before allocation and share byte-copying and MIME assignment helpers.
Native payload memory tests
test/js/bun/util/heap-snapshot.test.ts
Tests validate memory estimates for multipart File objects, forwarded FormData files, and WebSocket Blobs.
Screenshot Blob memory tests
test/js/bun/webview/*
Screenshot tests compare native PNG Blob memory usage with equivalent byte-backed Blobs.

Possibly related PRs

Suggested reviewers: jarred-sumner, sosukesuzuki

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The description identifies related follow-up work in PRs #37667 and #37656 and explains why those paths remain separate.
Out of Scope Changes check ✅ Passed The implementation remains focused on native Blob memory accounting and targeted regression tests, with unrelated paths explicitly excluded.
Title check ✅ Passed The title clearly summarizes the main change: native Blob bindings now report estimated size to the garbage collector.
Description check ✅ Passed The description clearly explains the problem, fix, scope, and verification, although it does not use the template headings exactly.

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Aug 12, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: reproduced on bun 1.4.0 and on main with the snippet in the description (estimateShallowMemoryUsageOf returns 48 for a 1 MiB multipart File, 81 for the parsed FormData, 48 / 56 for a WebSocket blob message and its MessageEvent, 48 for a webview screenshot). The fix is in Blob__dupe / Blob__fromBytes* / Blob__fromMmapWithType in src/runtime/webcore/Blob.rs; the new assertions in test/js/bun/util/heap-snapshot.test.ts and the two webview screenshot tests fail on the unfixed binary and pass with the fix.

Related PRs for the other producers with the same symptom: #37667 (new File(...)) and #37656 (Bun.Image#blob() / the by-value JsClass::to_js). Neither touches the functions changed here; the description ends with where the three should be consolidated afterwards.

CI is green on the final revision (Buildkite build 92978, ASAN lane included) and every review thread is resolved. Ready for a maintainer.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — focused GC accounting fix that fills reported_estimated_size at the four C++ binding entry points, with matching tests.

What was reviewed:

  • new_for_bindings computes the size on the stack Blob before Blob::new heap-promotes it; the Cell value moves with the struct, so Blob__estimatedSize and reportExtraMemoryAllocated see it.
  • Content type is now set before size computation in the *WithType paths, so it's included in the estimate; the two duplicated stamping blocks collapse into set_content_type_from_cstr and the raw *blob derefs are gone.
  • Blob__fromBytes empty/null-ptr and non-unix Blob__fromMmapWithType fallback paths preserved.
  • Tests wire onerror/onclose to reject, use port: 0 and using, and bound the estimate at 1×–2× rather than an exact value.
Extended reasoning...

Overview

The PR touches two files: src/runtime/webcore/Blob.rs (four extern "C" exports refactored to route through a new new_for_bindings helper that calls calculate_estimated_byte_size() before Blob::new; two new private helpers blob_from_bytes and set_content_type_from_cstr extracted from duplicated inline code; a stale doc comment on Blob__fromBytesWithType corrected) and test/js/bun/util/heap-snapshot.test.ts (a new nested describe with four tests covering multipart Request/Response parsing, re-appending a parsed File, and WebSocket binaryType = "blob").

Security risks

None. The only observable change is the value written into reported_estimated_size, which JSC reads for GC scheduling, visitChildren, and heap-snapshot reporting. No user-facing API surface, parsing, or validation changes. The refactor actually reduces unsafe surface: the previous code heap-promoted first and then dereferenced the raw *mut Blob to set the content type; the new code sets it on the stack value via &Blob + Cell::set before promotion.

Level of scrutiny

Low-to-medium. calculate_estimated_byte_size is an existing method that just sums a handful of field lengths and writes a Cell<usize>; calling it on a freshly built, exclusively-owned stack Blob on the JS thread is safe. The refactored exports are behaviorally identical apart from that one extra call — I compared each old body against the new helper composition (empty/null-ptr short-circuit in blob_from_bytes, the #[cfg(not(unix))] fallback in Blob__fromMmapWithType, and the BlobContentType::Owned(mime_slice.into()) stamping) and found no dropped side effects. The reordering that sets content type before size computation only makes the estimate more accurate.

Other factors

The PR description enumerates every consumer of these exports (WebCore::Blob toJS, FormData multipart entries, Blob__dupeFromJS, WebSocket blob mode, webview screenshots) and explains why sibling paths (new File, Bun.Image#blob, BuildArtifact) are handled in separate PRs — the scope boundary is deliberate and doesn't overlap. Tests follow harness conventions (bounded assertions, port: 0, using disposal, failure events wired to reject, try/finally cleanup) and the author confirmed they fail under USE_SYSTEM_BUN=1 and pass on the debug build alongside the existing Blob/FormData/WebSocket suites. The bug hunting system found nothing.

@robobun

robobun commented Aug 12, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 1:00 AM PT - Aug 12th, 2026

✅ @robobun, your commit 90b67e050ad03d6b3c935e35e904a3d38c64b2fb passed in Build #92978! 🎉


🧪   To try this PR locally:

bunx bun-pr 37697

That installs a local version of the PR into your bun-37697 executable, so you can run:

bun-37697 --bun

Comment thread src/runtime/webcore/Blob.rs Outdated
Comment thread src/runtime/webcore/Blob.rs
Comment thread src/runtime/webcore/Blob.rs
Comment thread src/runtime/webcore/Blob.rs

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/runtime/webcore/Blob.rs`:
- Around line 4257-4261: Update the Blob creation flow around new_for_bindings
and the toJSNewlyCreated path so the file configuration via Blob__setAsFile
occurs before Blob__create, or recalculate the estimated byte size after
stored_name is assigned. Add coverage using a file name whose length makes the
estimate change, and verify the returned wrapper reports the refreshed estimate.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 098b7524-fa30-4f06-bc40-5d55e60702fa

📥 Commits

Reviewing files that changed from the base of the PR and between e7abdf7 and 90b67e0.

📒 Files selected for processing (4)
  • src/runtime/webcore/Blob.rs
  • test/js/bun/util/heap-snapshot.test.ts
  • test/js/bun/webview/webview-chrome.test.ts
  • test/js/bun/webview/webview.test.ts

Comment thread src/runtime/webcore/Blob.rs
robobun added a commit that referenced this pull request Aug 12, 2026
…lone

Blob::new computing the size already covers those producers; #37667
(create-time reporting for File and S3File wrappers) and #37697 (content
type ordering in the Blob__from* exports) change those same lines and
apply independently of this branch, so this branch does not touch them.
calculate_estimated_byte_size stays pub for types that embed a Blob by
value and report it themselves.
@robobun

robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

Heads-up on overlap with #37656: its latest push computes reported_estimated_size inside Blob::new, which every pointer these exports return goes through, so the multipart / WebSocket / screenshot cases here are covered from that side too (your four heap-snapshot tests pass against that branch with only its change applied). #37656 deliberately does not touch Blob__dupe, the Blob__from* exports or the constructor line, so this PR applies independently in either order; the content type ordering in the *WithType exports and the tests here are not covered by #37656. Once both are in, new_for_bindings only repeats what Blob::new does and can be dropped, whichever lands second.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs; the earlier doc-comment nit I raised was addressed in 90b67e0. Because it refactors the unsafe FFI Blob constructors and is one of three coordinated PRs with a planned follow-up consolidation, a human look would still be worthwhile.

What was reviewed:

  • new_for_bindings writes reported_estimated_size on a stack-local Blob on the JS thread before Blob::new heap-promotes it — no marking-thread race.
  • blob_from_bytes / set_content_type_from_cstr are byte-for-byte extractions of the previous inline code; content type is now set before the size is cached, so it's included.
  • The unresolved CodeRabbit note on toJSNewlyCreated ordering is about pre-existing blob.cpp code and only affects a few bytes of filename, not the payload accounting this PR fixes.
  • New tests cover every touched export (Request/Response .formData(), re-appended File, WebSocket blob, both webview screenshot backends) and wire error/close to reject.
Extended reasoning...

Overview

The PR fixes GC memory accounting for Blobs created by the native C++ bindings rather than by new Blob(). Four extern "C" exports in src/runtime/webcore/Blob.rs (Blob__dupe, Blob__fromBytes, Blob__fromBytesWithType, Blob__fromMmapWithType) plus the JS Blob constructor are routed through a new two-line helper new_for_bindings that calls calculate_estimated_byte_size() before Blob::new(). Two copies of the content-type-stamping block are deduplicated into set_content_type_from_cstr, and the byte-copying prelude of the two fromBytes* variants into blob_from_bytes. Four new test cases in heap-snapshot.test.ts and one assertion each in the two webview screenshot tests cover every touched export.

Security risks

None. No new unsafe operations are introduced — the extracted helpers carry the exact safety contracts of the code they replace, and the raw-pointer dereferences that were previously done through (*blob) on a fresh heap allocation now happen on a stack &Blob before heap promotion, which is strictly safer. The value written is only read by JSC for GC scheduling and heap-snapshot reporting; it does not affect user-visible behaviour.

Level of scrutiny

Moderate. The change is mechanically simple (compute a cached size before heap-promoting; dedupe two identical blocks) and cannot regress runtime behaviour — the only observable effect is that estimateShallowMemoryUsageOf and JSC's extra-memory accounting now report the payload bytes instead of 0. The write to reported_estimated_size happens on the JS thread on an exclusively-owned stack value before Blob::new publishes it, so there is no interaction with the GC marking thread that reads the cache. That said, the touched functions are the FFI boundary between the C++ WebCore::Blob wrapper (used by FormData, WebSocket, MessageEvent, WebView) and the Rust Blob, and the PR is one of three coordinated changes (#37656, #37667) with a stated follow-up to move the computation into Blob::new itself — a maintainer should confirm the staging is what they want.

Other factors

  • My prior inline comment (drop the stale 'static from Blob__fromMmapWithType's safety doc) was applied in 90b67e0, and the comment-cop threads were resolved by shortening the new_for_bindings doc to one line.
  • The first CI build (#92698) was green including ASAN; the pushes since then only touched doc comments and added the screenshot assertions.
  • CodeRabbit left an unresolved note that toJSNewlyCreated in blob.cpp calls Blob__create before Blob__setAsFile, so the filename bytes aren't in the cached estimate on that path. That ordering is pre-existing C++ this PR doesn't touch, the underlying Rust Blob on that path now has its payload counted (via Blob__fromBytes → new_for_bindings), and the missed filename is a handful of bytes — the PR description already names moving the computation into Blob::new as the follow-up that closes this. I don't consider it a blocker but flagging it as the one open thread.
  • Test quality is good: port: 0, using for the server, error/close events wired to reject the awaited promise, bounds asserted as [1x, 2x) of a 1 MiB payload so they can't pass vacuously.

@robobun

robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

One correction to the above: review on #37656 pointed out the constructor's own calculate_estimated_byte_size() call is dead once Blob::new computes, so 9903397 removes it after all. That is the one hunk the two diffs share (this PR turns the same two lines into Ok(new_for_bindings(blob))); whichever lands second resolves it by keeping Ok(Blob::new(blob)). Everything else still applies independently.

@robobun

robobun commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator Author

Heads-up from #38562: Blob__create and the generated Blob constructor now report Blob__newlyAllocatedSize(ptr) at wrapper creation (only the Blob struct when the store is shared) and keep using Blob__estimatedSize for visitChildren / memoryCost / estimateShallowMemoryUsageOf. Computing the cached estimate in Blob__dupe / Blob__fromBytes* as this PR does is compatible: it fixes what the FormData entry, MessageEvent and each wrapper report as retained, while #38562 keeps formData.get() / event.data from reporting the payload as a fresh allocation per call. The assertions here are all on the retained side, so they should pass unchanged on top of it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant