bundler: copy file/wasm/napi/sqlite assets byte-for-byte when they start with a BOM

[robobun]

Problem

• bun build / Bun.build rewrite assets whose first bytes look like a byte-order mark: an asset starting with EF BB BF is emitted without those three bytes, and one starting with FF FE (a UTF-16LE text file, e.g. a .reg or an Excel "Unicode text" export imported through the file loader) is emitted re-encoded as UTF-8. The content hash in the asset name is computed from the rewritten bytes too.
• Affects every loader that emits the input file itself: file, wasm, napi, sqlite (embed: "true"). Repro on bun 1.4.0: printf '\xef\xbb\xbfhello' > a.bin bundled with --loader .bin:file produces a 5-byte a-*.bin (68 65 6c 6c 6f); printf '\xff\xfeh\0i\0' > b.bin produces a 2-byte b-*.bin (68 69).
• Cause: ParseTask::get_code_for_parse_task_without_plugins (src/bundler/ParseTask.rs:1454) reads every input through cache::Fs::read_file_with_allocator, and the readers behind it (finish_arena_contents and the three arms of read_file_with_handle_impl in src/resolver/fs.rs) unconditionally run BOM::detect + strip/transcode. That is right for text that is about to be parsed and wrong for bytes that are copied into the output: process_files_to_copy (src/bundler/bundle_v2.rs:4285) writes source.contents as the asset, so the rewritten buffer is what lands on disk.

Fix

• read_file_with_allocator (and the read_file_contents* helpers under it) take a new bun_resolver::fs::BomHandling { Convert, Keep }; the four BOM sites in fs.rs go through BomHandling::detect, which returns None for Keep. The fixing line is the Fs::BomHandling::for_loader(loader) argument in ParseTask.rs; the previously unused _loader parameter already carried the loader there.
• Loader::is_binary() (src/ast/loader.rs) is the classification: file, wasm, napi, sqlite, sqlite_embedded, base64, dataurl read verbatim; everything else keeps the current stripping/transcoding. It is an exhaustive match so a new loader has to be classified. base64/dataurl are listed because they encode the raw bytes once implemented (bundler: implement dataurl and base64 loaders #36327, runtime + --no-bundle: implement base64/dataurl loaders #36334; the latter currently re-reads the file itself to get around this reader); in the bundler today they ignore the contents, so listing them changes nothing yet.
• Why this is correct: the BOM conversion exists so parsers see UTF-8 text; a copied asset has no parser, and the bytes on disk are the output contract. esbuild's file loader emits all three repro inputs unchanged (8, 6 and 6 bytes with esbuild 0.18.6; output in the details block). Text reads are unchanged: package.json, tsconfig.json, bun build CSS entry points, and the runtime Transpiler::parse path all pass Convert (the runtime path derives it from its loader like the bundler does; only text loaders, plus wasm, whose magic \0asm cannot collide with a BOM, reach it).
• Not changed: bun build --no-bundle copies assets with a file copy (build_copied_file_output, see bun build --no-bundle: write output files when --outdir is set #35644) and the runtime file loader only exports the path, so neither went through this reader. Plugin-provided contents (onLoad, in-memory files) never did either.
• Verified: test/bundler/bundler_loader.test.ts ("BOM-prefixed inputs"): loader-copy-keeps-bom-bytes-{api,cli} bundle all four copying loaders with a UTF-8-BOM and a UTF-16LE-BOM payload and compare the emitted bytes with the input; all 8 assets come out rewritten on USE_SYSTEM_BUN=1 (1.4.0) and byte-exact with this build. loader-text-converts-bom pins the unchanged side: .txt and .js inputs with either BOM still come through stripped/transcoded.
• Also ran with this build: the rest of bundler_loader.test.ts (53 pass), bundler_bun.test.ts (sqlite embed), bundler_files.test.ts, native-plugin.test.ts (the fetchSourceCode path goes through the same function; 19 pass, 1 pre-existing skip), and the repro above (a-*.bin is 8 bytes, b-*.bin is 6).

Background

• BOM (byte-order mark): the optional bytes EF BB BF (UTF-8) or FF FE (UTF-16LE) at the start of a text file that declare its encoding. Bun's file reader (bun_core::strings::BOM) detects these two and either drops the UTF-8 marker or transcodes the UTF-16LE body to UTF-8 so the JS/JSON/CSS/... parsers only ever see UTF-8. Nothing distinguishes a BOM from arbitrary binary data that happens to start with the same bytes; only the loader knows whether the file is text.
• cache::Fs::read_file_with_allocator (src/resolver/lib.rs): the one file reader shared by the resolver (package.json, tsconfig.json), the runtime transpiler, and the bundler's parse tasks. It handles fd reuse and the per-worker arena, which is why the fix is a parameter on it rather than a separate raw read in the bundler.
• ParseTask (src/bundler/ParseTask.rs): the per-input-file unit of work in the bundler. For copying loaders its "AST" is just a placeholder string for the asset URL; the file bytes are kept as source.contents and later written out by process_files_to_copy under a content-hashed name.

Repro before/after

$ printf '\xef\xbb\xbfhello' > a.bin; printf '\xff\xfeh\x00i\x00' > b.bin
$ printf 'import a from "./a.bin"; import b from "./b.bin"; console.log(a, b);' > e.js
$ bun build ./e.js --loader .bin:file --outdir=dist

# bun 1.4.0
a-hvs6h2nf.bin  5 bytes    68 65 6c 6c 6f
b-tezyx107.bin  2 bytes    68 69

# this branch
a-rs58fy5c.bin  8 bytes    ef bb bf 68 65 6c 6c 6f
b-z0j5q0bj.bin  6 bytes    ff fe 68 00 69 00

# esbuild 0.18.6: esbuild e.js --bundle --loader:.bin=file --outdir=esb
a-WQADS3YA.bin  8b         ef bb bf 68 65 6c 6c 6f
b-G5LK74FN.bin  6b         ff fe 68 00 69 00

A third input starting with FE FF (UTF-16BE) was already copied unchanged by bun because BOM::detect does not recognize that marker.

[coderabbitai]

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 11 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: d1f8ec40-3819-49f2-8af0-02c8e36237ce

📥 Commits

Reviewing files that changed from the base of the PR and between b7a0431 and 8d99e6b.

📒 Files selected for processing (8)

• src/ast/loader.rs
• src/bundler/ParseTask.rs
• src/bundler/transpiler.rs
• src/resolver/fs.rs
• src/resolver/lib.rs
• src/resolver/package_json.rs
• src/resolver/resolver.rs
• test/bundler/bundler_loader.test.ts

---

_{Comment @coderabbitai help to get the list of available commands.}

[robobun]

Status: fix pushed, waiting on CI.

Reproduced on bun 1.4.0 with test/bundler/bundler_loader.test.ts ("BOM-prefixed inputs"): under USE_SYSTEM_BUN=1 all 8 copied assets (file, wasm, napi, sqlite embed, each with a UTF-8 BOM and a UTF-16LE BOM payload) come out rewritten (68 65 6c 6c 6f / 68 69); with this branch they are byte-identical to the inputs. The text-loader case in the same block passes on both builds, which is intended: it pins the behavior this PR leaves alone.

[claude]

LGTM — mechanical parameter threading with an exhaustive loader classification, and the tests pin both sides.

What was reviewed:

• Verified all five read_file_with_allocator callers are updated; text-only sites (package.json, tsconfig, CSS entry) pass Convert, so behavior is unchanged there.
• Confirmed read_file_with_handle_impl/read_file_contents* have no other direct callers outside the updated cache::Fs wrapper.
• Checked the Loader::is_binary() classification against should_copy_for_bundling — the two agree on the copying loaders, plus base64/dataurl which encode raw bytes.
• Tests cover all four copying loaders × both BOM prefixes × api/cli, and separately pin that .txt/.js inputs still get stripped/transcoded.

Extended reasoning...

Overview

Threads a BomHandling { Convert, Keep } parameter through the resolver's shared file reader (read_file_with_allocator and the read_file_contents*/read_file_with_handle_impl helpers under it) so callers can opt out of BOM stripping/UTF-16LE transcoding. Adds Loader::is_binary() (an exhaustive match) as the classification, and passes BomHandling::for_loader(loader) at the two loader-aware call sites (ParseTask and the runtime Transpiler::parse). The other three callers (package.json, tsconfig.json, CSS entry) pass Convert explicitly, preserving current behavior. Adds a bundler test covering all four copy-to-outdir loaders × two BOM prefixes × api/cli backends, plus a companion test pinning that text loaders still convert.

Security risks

None. This is a data-fidelity fix to the bundler's asset copy path; no untrusted-input parsing, auth, or crypto is touched.

Level of scrutiny

Moderate. The shared reader in src/resolver/fs.rs is used by the resolver, runtime transpiler, and bundler, so a mistake here has broad reach. However, the change is a strictly additive parameter with the four in-body BOM::detect sites replaced by bom_handling.detect, and every pre-existing call site either passes Convert (identical to the old unconditional path) or derives it from a loader that was already in scope. I grepped for all callers of read_file_with_allocator, read_file_with_handle_impl, read_file_contents, and read_file_contents_in_arena — every one is updated in this diff and there are no other direct callers.

Other factors

• is_binary() is an exhaustive match, so a future Loader variant will fail to compile until classified — meets the review guideline about atomically updating consumers.
• The classification agrees with should_copy_for_bundling() on the copying loaders and additionally lists Base64/Dataurl (which encode raw bytes and, per the PR description, don't consume the reader's contents in the bundler today, so listing them changes nothing yet).
• The runtime-transpiler call site at transpiler.rs:1444 now uses for_loader(loader); the PR description notes only text loaders plus wasm (whose \\0asm magic cannot collide with a BOM) reach it, so this is behavior-preserving there.
• Tests are strong: byte-exact hex comparison of eight emitted assets (fails on USE_SYSTEM_BUN=1 per the description), and a positive test that .txt/.js still get BOM-stripped/transcoded so the unchanged side is pinned.
• esbuild parity is documented in the description.

[robobun]

^{Updated 6:57 AM PT - Aug 13th, 2026}

❌ @robobun, your commit 8d99e6b has 1 failures in Build #94374 (All Failures):

• test/js/bun/dns/resolve-dns.test.ts - code 1 on 🍎 14 aarch64

---

🧪   To try this PR locally:

bunx bun-pr 38103

That installs a local version of the PR into your bun-38103 executable, so you can run:

bun-38103 --bun