node-fallbacks: embed browser polyfills zstd-compressed#31456
Conversation
|
Updated 8:01 PM PT - May 26th, 2026
✅ @sosukesuzuki, your commit 2c47eb50b446c3c597fdf97021099d8ab0a502f5 passed in 🧪 To try this PR locally: bunx bun-pr 31456That installs a local version of the PR into your bun-31456 --bun |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (4)
WalkthroughThis PR adds Zstandard compression to the node fallbacks build pipeline. Generated fallback scripts are now compressed at build time and decompressed lazily at runtime, with both ChangesNode Fallbacks Zstd Compression
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
|
This PR may be a duplicate of:
🤖 Generated with Claude Code |
There was a problem hiding this comment.
I didn't find any issues, but this changes how embedded resources are loaded in release builds across the build system and resolver, so it's worth a human sign-off on the approach.
Extended reasoning...
Overview
This PR compresses the ~1 MB of browser polyfill JS (node-fallbacks/*.js) with zstd-19 at codegen time and embeds the compressed bytes in release builds, decompressing lazily on first access via a per-module bun_core::Once<String>. It touches:
src/node-fallbacks/build-fallbacks.ts— emit a.js.zstalongside each bundled.jsscripts/build/codegen.ts— declare the.js.zstfiles as ninja outputssrc/resolver/node_fallbacks.rs— rewritecreate_source_code_getter!toinclude_bytes!the.zstand decompress on first call undercfg(bun_codegen_embed); debug path unchangedsrc/resolver/Cargo.toml/Cargo.lock— addbun_zstdworkspace dep
I verified that bun_core::Once::new() is const fn with get_or_init, bun_zstd::decompress_alloc returns Result<Vec<u8>, _>, and the new macro mirrors the existing runtime_embed_file! cfg-split pattern in src/bun_core/util.rs exactly (same #[allow(unexpected_cfgs)] + env!("BUN_CODEGEN_DIR") path construction).
Security risks
None identified. The compressed payloads are produced at build time from in-repo sources and embedded via include_bytes!; there's no untrusted input. Decompression failures panic with .expect(), which is appropriate for build-time-controlled data.
Level of scrutiny
Medium-high. The code is small and follows established patterns, and the author verified byte-identical bundler output plus passing tests on both debug and release paths. However, this is an architectural change to how a class of embedded resources is stored and loaded in the shipped binary, spanning the ninja build graph, codegen scripts, and a runtime macro. A maintainer should confirm the size/complexity tradeoff is wanted, that .expect() panic-on-decompress is acceptable in the bundler path, and whether this pattern should eventually generalize to other embedded files.
Other factors
- The
Once<String>static is per macro expansion (one per polyfill), so concurrent bundler threads requesting the same polyfill are correctly serialized byget_or_init. react-refresh.jsis intentionally left uncompressed (consumed by the bake dev server, not this resolver) and the codegen change correctly only adds.zstoutputs for thesources.nodeFallbacksset.- CI hasn't reported yet on the timeline.
Summary
Rust counterpart of #30347 (which targeted the old Zig implementation and is now stale).
The bundled
node-fallbacks/*.jsbrowser polyfills are embedded as plain-text strings in.rodata(~1 MB, 477 KB of which iscrypto.jsalone). They are only ever read when bundling with--target=browser, so everyone who never bundles for the browser pays the ~1 MB on disk for nothing.This PR compresses each polyfill with zstd-19 at codegen time (~85% ratio) and embeds the
.js.zstin release builds (bun_codegen_embed). The first access to a given module decompresses it into the heap and caches it for the process lifetime. Debug builds keep reading the uncompressed.jsfromBUN_CODEGEN_DIRat runtime, so JS-only edits still don't trigger a native rebuild.Size
Measured with the
btgprofile (Release + LTO, linux x64), both builds at the same base commit, same build directory:.rodata.textcrypto.jspolyfill in binaryPerformance
bun build --target=browser, hyperfine, warmup=3, runs=30, linux x64 (Xeon Platinum 8488C), Release+LTO builds:worst case — entry imports the 5 largest polyfills (crypto, stream, assert, zlib, http; all decompressed on the path):
no polyfills —
console.log("hello")entry: 5.1 ms vs 5.2 ms (1.01 ± 0.05, within noise).runtime startup —
bun -e 'console.log(1)': 10.1 ms vs 10.1 ms (1.00 ± 0.04). Decompression is lazy; it never runs unless a polyfill is actually requested.Bundler output is byte-identical before and after (verified for an entry importing all 23 polyfills and for the worst-case entry above).
Implementation
src/node-fallbacks/build-fallbacks.ts— write a.js.zstnext to each bundled.js(Bun.zstdCompressSync(..., { level: 19 })).scripts/build/codegen.ts— declare both.jsand.js.zstas ninja outputs of the codegen step (both feed the cargo edge's implicit inputs).src/resolver/node_fallbacks.rs—create_source_code_getter!embeds the.zstviainclude_bytes!undercfg(bun_codegen_embed)and lazily decompresses into a per-modulebun_core::Once<String>on first call; thecfg(not(bun_codegen_embed))(debug) path is unchanged.src/resolver/Cargo.toml— addbun_zstddependency.react-refresh.jsis unchanged since it's referenced from the bake dev server, not the fallback resolver.Test plan
bun bd test test/bundler/bundler_browser.test.ts— 12 pass / 0 fail (debug build, runtime-load path)bun build --target=browseroutput diff'd byte-identical against a pre-change build (all 23 polyfills)