Skip to content

node-fallbacks: embed browser polyfills zstd-compressed#31456

Merged
Jarred-Sumner merged 1 commit into
mainfrom
claude/node-fallbacks-zstd-embed
May 28, 2026
Merged

node-fallbacks: embed browser polyfills zstd-compressed#31456
Jarred-Sumner merged 1 commit into
mainfrom
claude/node-fallbacks-zstd-embed

Conversation

@sosukesuzuki

Copy link
Copy Markdown
Contributor

Summary

Rust counterpart of #30347 (which targeted the old Zig implementation and is now stale).

The bundled node-fallbacks/*.js browser polyfills are embedded as plain-text strings in .rodata (~1 MB, 477 KB of which is crypto.js alone). They are only ever read when bundling with --target=browser, so everyone who never bundles for the browser pays the ~1 MB on disk for nothing.

This PR compresses each polyfill with zstd-19 at codegen time (~85% ratio) and embeds the .js.zst in release builds (bun_codegen_embed). The first access to a given module decompresses it into the heap and caches it for the process lifetime. Debug builds keep reading the uncompressed .js from BUN_CODEGEN_DIR at runtime, so JS-only edits still don't trigger a native rebuild.

Size

Measured with the btg profile (Release + LTO, linux x64), both builds at the same base commit, same build directory:

before after Δ
stripped binary 74,713,200 B 73,910,384 B −802,816 B (−1.07%)
.rodata 22,036,416 B 21,232,448 B −803,968 B
.text 52,332,682 B 52,338,218 B +5,536 B (lazy-decompress getters)
crypto.js polyfill in binary 477,051 B 71,164 B

Performance

bun build --target=browser, hyperfine, warmup=3, runs=30, linux x64 (Xeon Platinum 8488C), Release+LTO builds:

worst case — entry imports the 5 largest polyfills (crypto, stream, assert, zlib, http; all decompressed on the path):

mean ± σ
before 33.9 ms ± 0.6 ms
after 34.5 ms ± 0.5 ms
ratio 1.02 ± 0.02 (≈ +0.6 ms once per build for ~700 KB of one-time zstd decompression)

no polyfillsconsole.log("hello") entry: 5.1 ms vs 5.2 ms (1.01 ± 0.05, within noise).

runtime startupbun -e 'console.log(1)': 10.1 ms vs 10.1 ms (1.00 ± 0.04). Decompression is lazy; it never runs unless a polyfill is actually requested.

Bundler output is byte-identical before and after (verified for an entry importing all 23 polyfills and for the worst-case entry above).

Implementation

  • src/node-fallbacks/build-fallbacks.ts — write a .js.zst next to each bundled .js (Bun.zstdCompressSync(..., { level: 19 })).
  • scripts/build/codegen.ts — declare both .js and .js.zst as ninja outputs of the codegen step (both feed the cargo edge's implicit inputs).
  • src/resolver/node_fallbacks.rscreate_source_code_getter! embeds the .zst via include_bytes! under cfg(bun_codegen_embed) and lazily decompresses into a per-module bun_core::Once<String> on first call; the cfg(not(bun_codegen_embed)) (debug) path is unchanged.
  • src/resolver/Cargo.toml — add bun_zstd dependency.

react-refresh.js is unchanged since it's referenced from the bake dev server, not the fallback resolver.

Test plan

  • bun bd test test/bundler/bundler_browser.test.ts — 12 pass / 0 fail (debug build, runtime-load path)
  • same test file run with the Release+LTO binary (exercises the embed + decompress path) — 12 pass / 0 fail
  • bun build --target=browser output diff'd byte-identical against a pre-change build (all 23 polyfills)
  • CI

@robobun

robobun commented May 27, 2026

Copy link
Copy Markdown
Collaborator
Updated 8:01 PM PT - May 26th, 2026

@sosukesuzuki, your commit 2c47eb50b446c3c597fdf97021099d8ab0a502f5 passed in Build #58317! 🎉


🧪   To try this PR locally:

bunx bun-pr 31456

That installs a local version of the PR into your bun-31456 executable, so you can run:

bun-31456 --bun

@coderabbitai

coderabbitai Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: d703eda9-ac4d-4df3-a478-ac2ef7206bfd

📥 Commits

Reviewing files that changed from the base of the PR and between 2148214 and 2c47eb5.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (4)
  • scripts/build/codegen.ts
  • src/node-fallbacks/build-fallbacks.ts
  • src/resolver/Cargo.toml
  • src/resolver/node_fallbacks.rs

Walkthrough

This PR adds Zstandard compression to the node fallbacks build pipeline. Generated fallback scripts are now compressed at build time and decompressed lazily at runtime, with both .js and .js.zst artifacts tracked by the build system.

Changes

Node Fallbacks Zstd Compression

Layer / File(s) Summary
Zstd runtime loading and dependency
src/resolver/Cargo.toml, src/resolver/node_fallbacks.rs
The create_source_code_getter! macro is updated to load and lazily decompress .zst files from BUN_CODEGEN_DIR under the bun_codegen_embed cfg, using bun_zstd::decompress_alloc and caching via bun_core::Once. The bun_zstd crate is added as a workspace dependency.
Build-time compression and output declaration
src/node-fallbacks/build-fallbacks.ts, scripts/build/codegen.ts
The build script compresses each generated output with Bun.zstdCompressSync at level 19, writing .zst twins alongside .js files. The codegen declares both .js and .js.zst outputs to the build system.
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: embedding browser polyfills with zstd compression.
Description check ✅ Passed The description provides comprehensive information exceeding template requirements: detailed rationale, size/performance metrics, implementation details, and test results.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

Copy link
Copy Markdown
Contributor

This PR may be a duplicate of:

  1. node-fallbacks: embed browser polyfills zstd-compressed #30347 - Same feature (zstd-compress node-fallback browser polyfills) by the same author, targeting the old Zig implementation; this PR is the Rust rewrite

🤖 Generated with Claude Code

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't find any issues, but this changes how embedded resources are loaded in release builds across the build system and resolver, so it's worth a human sign-off on the approach.

Extended reasoning...

Overview

This PR compresses the ~1 MB of browser polyfill JS (node-fallbacks/*.js) with zstd-19 at codegen time and embeds the compressed bytes in release builds, decompressing lazily on first access via a per-module bun_core::Once<String>. It touches:

  • src/node-fallbacks/build-fallbacks.ts — emit a .js.zst alongside each bundled .js
  • scripts/build/codegen.ts — declare the .js.zst files as ninja outputs
  • src/resolver/node_fallbacks.rs — rewrite create_source_code_getter! to include_bytes! the .zst and decompress on first call under cfg(bun_codegen_embed); debug path unchanged
  • src/resolver/Cargo.toml / Cargo.lock — add bun_zstd workspace dep

I verified that bun_core::Once::new() is const fn with get_or_init, bun_zstd::decompress_alloc returns Result<Vec<u8>, _>, and the new macro mirrors the existing runtime_embed_file! cfg-split pattern in src/bun_core/util.rs exactly (same #[allow(unexpected_cfgs)] + env!("BUN_CODEGEN_DIR") path construction).

Security risks

None identified. The compressed payloads are produced at build time from in-repo sources and embedded via include_bytes!; there's no untrusted input. Decompression failures panic with .expect(), which is appropriate for build-time-controlled data.

Level of scrutiny

Medium-high. The code is small and follows established patterns, and the author verified byte-identical bundler output plus passing tests on both debug and release paths. However, this is an architectural change to how a class of embedded resources is stored and loaded in the shipped binary, spanning the ninja build graph, codegen scripts, and a runtime macro. A maintainer should confirm the size/complexity tradeoff is wanted, that .expect() panic-on-decompress is acceptable in the bundler path, and whether this pattern should eventually generalize to other embedded files.

Other factors

  • The Once<String> static is per macro expansion (one per polyfill), so concurrent bundler threads requesting the same polyfill are correctly serialized by get_or_init.
  • react-refresh.js is intentionally left uncompressed (consumed by the bake dev server, not this resolver) and the codegen change correctly only adds .zst outputs for the sources.nodeFallbacks set.
  • CI hasn't reported yet on the timeline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants