Skip to content

bundler: list metafile inputs in reachable-files order and CSS chunks in entry point order - #36974

Open
robobun wants to merge 1 commit into
mainfrom
farm/e9ef9e0d/metafile-deterministic-inputs
Open

robobun wants to merge 1 commit into
mainfrom
farm/e9ef9e0d/metafile-deterministic-inputs

Conversation

@robobun

@robobun robobun commented Aug 5, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • The same sources build to different output each time. On Bun 1.4.2 one server build gives 8 contents of out/server.js in 40 builds and 5 --compile executables in 8. meta.json of five files has 2 contents in 50 builds.
  • MetafileBuilder::generate (MetafileBuilder.rs:247) lists inputs by source index. compute_chunks (computeChunks.rs:516) sorts CSS chunks, the output stylesheets, by a hash of source indices. A source index follows the arrival of parse results.
  • No user report. Repeated builds found it.

Fix

  • generate walks reachable_files, as LinkerGraph.rs:242 says and as esbuild does.
  • compute_chunks keeps CSS chunks in the order the entry points made them. CssImportOrder::hash is unchanged.
  • Verified: test/bundler/metafile.test.ts, 11 new tests. 10 fail on main every run.
  • Self-reviewed: 14 concerns raised, 13 addressed. A scan-order shuffle for tests is a follow-up.

Background

  • reachable_files is the linker's own order: the import graph walked from the entry points.
  • The position of a CSS chunk reaches HTML import manifests, executables and [hash] names.
  • The first design hashed paths into the chunk key. Two modules with one path then shared a stylesheet.

Downsides

Notes

Request. Repeated builds of five files gave 2 meta.json files in 50 builds. The request was a rebase of this PR and a test of two sources with one path. That test showed a defect in the first design, so this push replaces the design.

Repro, five plain files.

printf 'import { a } from "./a.js";\nimport { b } from "./b.js";\nconsole.log(a + b);\n' > entry.js
printf 'import { x } from "./x.js";\nexport const a = x;\n' > a.js
printf 'import { y } from "./y.js";\nexport const b = y;\n' > b.js
printf 'export const x = 1;\n' > x.js
printf 'export const y = 2;\n' > y.js
for i in $(seq 50); do bun build entry.js --outdir=out --metafile=meta.json >/dev/null; sha1sum < meta.json; done | sort | uniq -c

main lists entry.js b.js a.js x.js y.js or entry.js b.js a.js y.js x.js. This PR lists x.js a.js y.js b.js entry.js in 50 of 50 builds. esbuild 0.25.1 gives that order too.

Repro, emitted bytes. server.js imports page.html. The script of the page has four import("./lazyN.js"), and each lazyN.js imports lazyN.css. Build with bun build server.js --target=bun --splitting --outdir=out.

Same sources, one directory Bun 1.4.2 main 57fe602 this PR
out/server.js, 40 builds 8 contents 7 to 10 1
server-[hash].js, 24 builds 1 name, 6 contents 1 name, 7 contents 1 name, 1 content
--compile --splitting executable, 8 builds 5 3 to 5 1
meta.json of the five plain files, 50 builds 2 2 1

Why it matters without a report. docs/bundler/executables.mdx:378 says that builds stay reproducible. #42151 and #43811 added tests that compare two executables byte for byte. Their fixtures have no stylesheet. A full-stack server with two or more CSS chunks breaks that promise on main.

Why the first design is gone.

  • It hashed the path of each file into the CSS chunk key. In bun build server.ts --target=bun, server.ts and the page it imports both use shared.css. main writes 5 files. The first design wrote 4, and the page linked ./server.css, the stylesheet of the server graph.
  • It sorted inputs by path with the source index as tie-break. Two records with one path still changed places.
  • Its docs section promised the same bytes in any directory. That is false today: a root-level index.js takes the name of its default export from the directory (alpha_default, beta_default).
  • Also weighed: hash stable_source_indices into the key and keep the sort. One unrelated new file then moves the CSS outputs, and the hunk touches the lines of bundler: compare CSS import orders before two entry points share a stylesheet #46357.

History. inputs changes order on 1.3.9 too. The CSS part is newer. Four JS entry points with one stylesheet each give 1 outputs order in 30 builds on 1.3.9 and 6 on 1.3.10. #27040 gave those chunks a key that hashes source indices. The sort by key was already there.

What moves.

  • inputs: the entry point was the first key. Each file now comes after the files it imports, and the entry point is last.
  • CSS chunks in outputs, in Bun.build().outputs, in the CLI table and in the manifest of an HTML import: entry point order. Over 42 shapes with 40 builds each, this PR has one order in 42 shapes and main in 22. Each of those 22 gets entry point order. Two CSS entry points keep their order. bun build d.css b.css a.css c.css lists d a b c on main and d b a c here.
  • One [hash] name. A chunk hashes the chunks it reaches in chunk order. A page whose script loads four stylesheets with import() is page-vhb6bsa8.js on main and page-yzwj0y4w.js here. With --css-chunking that name depends on the directory on main (page-112k66w8.js, page-63ja3jvn.js). Here it is one name.
  • JS chunks keep the order they have on main. Past eight entry points that order is not entry point order. Ten JS entry points with one stylesheet each list JS e0 e8 e9 e1 .. e7 and CSS e0 .. e9.
  • esbuild 0.25.1 lists inputs in the same order. It lists CSS entry points in argv order too.

Tests. 11 new tests in one describe.

  • 4 use an onLoad plugin that holds a file until the bundler asks for an import of another file. That fixes the order in which parse results arrive, and each test builds in two orders.
  • 5 run the CLI and assert an exact order that main does not give. One of them builds in three directories.
  • 1 compiles an executable in two parse orders. It runs on release builds only: one compile takes 22 s on a debug build.
  • 1 checks that a stylesheet in the server graph and the browser graph stays two chunks. On main its chunk count holds and its order depends on the directory.
  • On Bun 1.4.2 and on a release build of main, 10 of the tests fail in 10 of 10 runs. A debug build of main fails 9 and skips the compile test. All 11 pass in 10 of 10 runs on a release build of this PR. The 10 that a debug build runs pass in 10 of 10 runs there.
  • With the files_live lines of generate deleted, 3 of the tests fail. With a walk that skips one file of a chunk, the new debug assertion stops the build.

Measurements. Release builds of 57fe602 with and without this diff, Linux x64.

  • 50 builds of the five files: 1 meta.json and 1 meta.md (main 2 and 2).
  • 41 project-and-flag sets, 4 builds in each of two directories. One meta.json in 36 sets (main 2 to 5). One outputs order in 36 (main 21 to 26). The other 5 sets are the server build with two pages. No bundle name or byte differs from main. No inputs or outputs record differs from main.
  • Two forced parse orders: 1 inputs order and 1 CSS chunk order (main 2 and 2).
  • A plugin namespace with virtual modules, debug builds: 1 inputs order in 3 builds (main 3), the same 9 records.
  • One stylesheet in the server graph and the browser graph: 5 output files, 2 stylesheets (main 5 and 2).
  • compute_chunks with 4 CSS chunks, allocator calls on the bundle thread counted with gdb: 135 allocations and 71 frees (main 138 and 72). The function has 46,217 bytes (main 46,765).
  • generate on the five files: 56 allocator calls, as on main. 35,875 instructions (main 35,826). The function has 9,301 bytes (main 9,246).
  • text of the binary: 107,559,600 bytes (main 107,560,112).
  • process_resolve_queue and on_parse_task_complete have the same instructions as on main: 486 and 3,661 disassembly lines, 0 differ when absolute addresses are masked.
  • valgrind, perf, strace and bloaty are not installed on this machine. The counts come from gdb, size, nm and objdump.

Cost for a build that never had the problem. A build with no metafile and no CSS chunk runs none of the changed lines. A metafile build runs 49 more instructions in generate on the five files, with no new allocation and no sort. A build with CSS chunks loses two arena slices and one sort in compute_chunks.

Not fixed. Both hunks take their order from the entry point list. That list still follows arrival in three cases.

Questions for a maintainer.

Self-review. 14 concerns raised, 13 addressed. Not done: a test-only switch that shuffles scan order, so that a suite finds the next reader of source indices. It touches the scan phase and belongs in its own PR.

Earlier code. The compute_chunks hunk is the one on branch robobun/fa954c11/closed-entry-point-list.

Suites run on a debug build. On the final diff: test/bundler/metafile.test.ts (75 pass, 1 skip) and 18 other files: esbuild/{metafile,css,splitting,loader}, bundler_{html,html_server,naming,plugin,bun,loader,files,defer}, html-import-manifest, css/css-modules, bun-serve-html, bun-serve-html-manifest, bun-serve-html-entry and bake/dev-and-prod. On the same two hunks before the last rebase, 24 more: bundler_{splitting,edgecase,browser,regressions,compile_splitting}, esbuild/dce, bun-build-api, cli, four files in css/, bake/dev/{css,html} and 10 regression files. 7 tests time out in bun-build-api, cli and bundler_compile_splitting (--compile, bytecode and a 180 s loop). The same 7 time out on a debug build of main on this machine.


no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/bundler/metafile.test.ts

@coderabbitai

coderabbitai Bot commented Aug 5, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 10b0f39 and 5a2cff7.

Files selected for processing: 3
  • src/bundler/linker_context/MetafileBuilder.rs
  • src/bundler/linker_context/computeChunks.rs
  • test/bundler/metafile.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.

Walkthrough

The bundler now emits metafile inputs in reachable-file order and retains CSS chunk insertion order when assigning chunk positions and JavaScript CSS references. New tests vary plugin load completion order and compare metafile records, CSS outputs, embedded HTML manifests, and compiled executable hashes.

Changes

Deterministic bundler ordering

Layer / File(s) Summary
Metafile input ordering
src/bundler/linker_context/MetafileBuilder.rs, test/bundler/metafile.test.ts
Metafile inputs are emitted by traversing reachable files. A debug assertion checks seen-source bits for reachable files. Tests compare input ordering across builds and output formats.
CSS chunk order and output determinism
src/bundler/linker_context/computeChunks.rs, test/bundler/metafile.test.ts
CSS chunks retain insertion order, and JavaScript CSS references use the first CSS chunk index. Tests compare CSS ordering, output records, embedded manifests, and compiled executable hashes.

Suggested reviewers: jarred-sumner, alii

Priority: Normal

Merge Risk: Minimal · up to 5a2cf

The ordering changes have no identified merge-blocking defect; the compiled-executable test follows an existing bundler-test timeout pattern.

Pre-merge checks: Passed 4
Check name Status Explanation
Title check Passed The title clearly summarizes both primary changes: deterministic metafile input ordering and CSS chunk ordering by entry point.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description is complete and relevant. It explains the problem, fix, scope, known limitations, performance impact, and verification results. It does not use the exact template headings, but it prov…
Full details: Description check

Explanation

The description is complete and relevant. It explains the problem, fix, scope, known limitations, performance impact, and verification results. It does not use the exact template headings, but it provides the required information in equivalent sections.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the claude label Aug 5, 2026
Comment thread test/bundler/metafile.test.ts Resolved

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code change is straightforward and looks correct, but the new "Reproducible builds" docs section establishes a formal user-facing guarantee (byte-identical output across directories, no timestamps, no absolute paths, content-derived hashes) that a maintainer should sign off on — especially since imports[].path still leaks absolute paths pending #34534. There's also an unresolved comment-cop note on the 4-line comment at MetafileBuilder.rs:232 (I think it's a false positive — the comment explains a non-local invariant, not a workaround — but you may want to address it).

What was reviewed:

  • The sort in generate() preserves all existing filter guards (seen_sources, RUNTIME, empty text/pretty) and uses index as tie-break for equal paths.
  • path.pretty is &[u8]; byte-order .cmp() matches the JS .sort() in the test for the ASCII fixture paths.
  • buildIn() now drains stdout alongside stderr (earlier nit addressed in 2abde25).
Extended reasoning...

Overview

Three files: MetafileBuilder.rs replaces the source-index-order iteration with collect → sort-by-pretty-path → emit; test/bundler/metafile.test.ts adds two determinism tests (cross-directory byte-identity via CLI, sorted-inputs via Bun.build); docs/bundler/index.mdx adds a new "Reproducible builds" section.

Security risks

None. This changes JSON key emission order in build metadata and adds documentation. No untrusted input parsing, no auth/crypto/permissions.

Level of scrutiny

The Rust change is low-risk and mechanical — the same filter guards are applied, only iteration order changes, and JSON key order is semantically meaningless to consumers. Tests are well-constructed (test.concurrent, tempDir, all pipes drained, both CLI and JS API surfaces covered, fixture designed so sorted order differs from discovery order).

The docs change is what tips this to human review: it documents a formal reproducibility contract (no timestamps, no absolute paths in sourcemap sources/metafile keys, content-derived hashes including debugId, deterministic ordering) that constrains future bundler behavior and that users will rely on. The PR description says bundles/sourcemaps/assets are "already byte-identical" but I can't independently verify that holds across all bundler configurations (splitting, plugins, --compile, HTML entrypoints, etc.). The claim is carefully scoped ("metafile's inputs and outputs keys", --compile explicitly excluded), but publishing a guarantee is a product decision a maintainer should confirm.

Other factors

  • My earlier nit (stdout not drained in buildIn()) was fixed in 2abde25 and the thread is resolved.
  • The comment-cop bot flagged the 4-line comment above ordered_inputs. I read it as a false positive: the comment explains why index order is non-deterministic (a fact about process_resolve_queue in another file), not a workaround justification. But it is unresolved on the PR.
  • The stripDirPrefix helper in the test is a deliberate accommodation for #34534 (absolute paths in imports[].path), which the PR description calls out; the two fixes are designed to land independently.
  • No CODEOWNERS cover these files.

Comment thread src/bundler/Chunk.rs Resolved
@robobun

robobun commented Aug 5, 2026 •

Copy link
Copy Markdown
Collaborator Author

Update, 2026-10-27. This comment describes the first design of this PR. That design is replaced: the path hash for CSS chunks merged the stylesheets of two modules that share a path, and the docs section is not in the PR any more. See the description for the current design.


Follow-up from self-review: probing multi-entry builds surfaced a second source of metafile nondeterminism, worse than the first. With multiple entry points importing CSS (including multi-page HTML builds), the outputs section listed CSS chunks in a different order on every run, even in the same directory. CSS chunks are deduplicated and ordered by a hash over their import order entries, and that hash used raw source indices, which are assigned in scheduling-dependent order.

fca392a fixes it by hashing the file's pretty path instead, so chunk identity and order become a function of the input files alone, and adds multi-entry CSS and multi-page HTML determinism tests (both fail on main). The docs section is also scoped more precisely: --format=cjs inlining of import.meta.url (it bakes the absolute source path into the bundle), --compile executables, and --bytecode artifacts are explicitly outside the guarantee.

The PR description now covers both mechanisms. CSS, HTML, and splitting suites pass: esbuild CSS (56), bundler_html (22), bundler_html_server, html-import-manifest, bundler_splitting.

Comment thread docs/bundler/index.mdx Resolved
Comment thread src/bundler/Chunk.rs Resolved
Comment thread src/bundler/Chunk.rs Resolved

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/bundler/Chunk.rs`:
- Around line 1426-1436: Update the layer hashing loop in the chunk-key
generation logic to encode each layer component with its length before its raw
bytes, so embedded separators or NULs cannot create collisions between distinct
parsed identifiers. Keep layer boundaries unambiguous and preserve deterministic
ordering, then add a regression test covering equivalent-looking paths such as
escaped-dot components in different positions and assert they produce distinct
CSS chunk map keys.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 555ccc60-3625-4df2-9c3a-8c5c91c7ee34

📥 Commits

Reviewing files that changed from the base of the PR and between 1f606e6 and 93c9565.

📒 Files selected for processing (5)
  • docs/bundler/index.mdx
  • src/bundler/Chunk.rs
  • src/bundler/linker_context/MetafileBuilder.rs
  • src/bundler/linker_context/computeChunks.rs
  • test/bundler/metafile.test.ts

Comment thread src/bundler/Chunk.rs Resolved
Comment thread src/bundler/Chunk.rs Resolved
@robobun

robobun commented Aug 5, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: reworked on 2026-10-27. Ready for review.

How it was reproduced.

  1. Five plain files: entry.js imports a.js and b.js, a.js imports x.js, b.js imports y.js. Run bun build entry.js --outdir=out --metafile=meta.json 50 times. main writes 2 different meta.json files. This PR writes 1.
  2. Emitted bytes: server.js imports page.html. The script of the page has four import() of modules that each import a stylesheet. Run bun build server.js --target=bun --splitting --outdir=out 40 times. out/server.js has 8 contents on Bun 1.4.2, 7 on main and 1 with this PR.

The description has the full numbers, the orders that move, and what is not fixed.

PR: #36974

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All earlier review feedback has been addressed and this pass found no issues. Deferring to a human for two sign-offs: the new Reproducible builds docs section states a public byte-identity contract for bun build output, and CssImportOrder::hash now derives CSS chunk identity from pretty paths instead of source indices — both look correct and are covered by the new determinism tests, but they are product/behavioral commitments rather than mechanical fixes.

What was reviewed

  • Metafile inputs sort in MetafileBuilder::generate — filter predicates preserved, tie-break on index keeps the sort total.
  • CssImportOrder::hash length-prefixed encoding — self-delimiting for layers, external paths, and source pretty paths; both call sites in computeChunks.rs updated.
  • New tests drain all pipes, use tempDir/test.concurrent, and strip only each build's own directory prefix before the byte-identity assertion.
Extended reasoning...

Overview

The PR makes bundler metafile and CSS chunk output deterministic across build directories. Three code changes: (1) MetafileBuilder::generate collects reachable input indices, sorts by path.pretty with index tie-break, then emits; (2) CssImportOrder::hash gains a sources: &[Source] parameter and hashes the file's pretty path instead of its scheduling-dependent source index, with every field length-prefixed so unescaped CSS identifiers cannot collide; (3) both hash call sites in computeChunks.rs pass sources. Four new determinism tests in test/bundler/metafile.test.ts and a new "Reproducible builds" section in docs/bundler/index.mdx.

Prior feedback resolved

My earlier inline comments (undrained stdout in buildIn, incomplete import.meta.* enumeration in the CJS docs caveat, and the pre-existing is_last index/length mismatch in the Layers hash arm) were fixed in 2abde25 and 93c9565. CodeRabbit's escaped-dot layer-component ambiguity was addressed in 10b0f39 by length-prefixing every count and byte string. All threads are resolved and this run's bug hunt found nothing.

Security risks

None identified. No untrusted input parsing was added; the hash is a non-cryptographic dedup key over bundler-internal data, and length-prefixing strictly reduces collision surface. The docs change is prose only.

Level of scrutiny

Medium-high. The sort in MetafileBuilder is mechanical, but the CssImportOrder::hash change redefines what makes two CSS chunks "the same" in compute_chunks' css_chunks map — a core bundler invariant. It looks correct (pretty paths are the same values used as metafile keys, and the encoding is now unambiguous), and the multi-entry CSS / multi-page HTML tests exercise the dedup path across runs and directories. Separately, the docs section is a user-facing reproducibility guarantee ("byte-identical bundles, sourcemaps, and assets on every run") with enumerated exceptions; per the repo's API-design guidance this is the kind of public contract a maintainer should explicitly agree to.

Other factors

Test quality is good: test.concurrent, tempDir with using, all pipes drained, no network, and the cross-directory assertion strips only each build's own absolute prefix so the remaining bytes must match exactly. The PR description shows the new tests fail on main (both debug+ASAN and release) and pass with the fix, and that the existing metafile/CSS/HTML/splitting suites pass. Given the docs contract and the chunk-identity semantics change, a maintainer sign-off is the right bar even though the implementation itself reads as correct.

@robobun

robobun commented Sep 13, 2026 •

Copy link
Copy Markdown
Collaborator Author

This PR also fixes a second symptom of the same defect: the key order of metafile.inputs changes between identical builds in one directory and one process.

The repro uses public APIs only. It has three entry points with import() edges into 14 modules, splitting: true and metafile: true, and it runs the same build in a loop.

build inputs key orders
release 1.4.2 (744846f) 7 different orders in 20 builds (2 of 2 runs)
canary 1.4.3-canary.1 (367d939, before the 1.4.3 tag) 8 different orders in 40 builds
main 5822b76 (debug, ASAN) 6 different orders in 10 builds
main 5822b76 merged with the first design of this PR (debug, ASAN) 1 order in 60 builds

Only the top-level inputs key order moves. outputs and every value under inputs are byte-identical between the runs. The entry points stay first, and the import groups of the three entry points permute.

The cause is the one this PR already names, with a second trigger. process_resolve_queue (src/bundler/bundle_v2.rs:6932) assigns a source index when the bundle thread handles a finished parse task. The entry points parse in parallel, so the order of the finished tasks follows thread scheduling. MetafileBuilder::generate writes inputs in source-index order. The description said that repeated builds in one directory are stable (8/8). That was wrong. One entry point with static imports only also changes order: five files give 2 orders in 50 builds.

Update, 2026-10-27. The paragraphs that were here pointed to the branch robobun/7e0d1f94/metafile-inputs-order. That branch held the first design of this PR (sort inputs by path, hash paths into the CSS chunk key). The first design merges the stylesheets of two modules that share a path, so the branch is deleted. The PR now has another design, see the description. It fixes the repro in this comment: 1 order in 20 builds.

Repro script
import fs from "node:fs"; import path from "node:path"; import os from "node:os";
const d = fs.mkdtempSync(path.join(os.tmpdir(), "mf-")); const N = 14;
for (let i = 0; i < N; i++) { let s = `export const v${i} = ${i};\n` + "// pad\n".repeat(((i * 37) % 11) * 300);
  for (const j of new Set([i + 1, (i * 5 + 3) % N, (i * 3 + 7) % N])) if (j < N && j > i) s += j % 2 ? `export const l${j} = () => import("./m${j}.js");\n` : `import * as s${j} from "./m${j}.js"; globalThis.k = s${j};\n`;
  fs.writeFileSync(path.join(d, `m${i}.js`), s); }
for (const e of [0, 1, 2]) fs.writeFileSync(path.join(d, `e${e}.js`), [0, 3, 5, 8, 11].map(i => `import("./m${(i + e) % N}.js");`).join("\n"));
const seen = new Map();
for (let run = 0; run < 20; run++) {
  const r = await Bun.build({ entrypoints: [0, 1, 2].map(e => path.join(d, `e${e}.js`)), splitting: true, metafile: true });
  const order = Object.keys(r.metafile.inputs).map(k => path.basename(k, ".js")).join(" ");
  seen.set(order, (seen.get(order) | 0) + 1);
}
for (const [o, n] of seen) console.log(n + "x", o);
console.log(seen.size === 1 ? "stable" : `UNSTABLE: ${seen.size} different key orders in 20 identical builds`);
fs.rmSync(d, { recursive: true, force: true });

Jarred-Sumner pushed a commit that referenced this pull request Sep 15, 2026
…].bytes (#42760)

### Problem
- In the JSON metafile (`bun build --metafile`, `Bun.build({ metafile:
true })`), `outputs[path].bytes` is smaller than the emitted file when
the chunk references another output. On 1.4.3 `m0.js` is 130 bytes on
disk, 98 in the metafile. `bun.d.ts` documents the field as "Size of the
output file in bytes".
- The cause is `generate_chunk_json`
(`src/bundler/linker_context/MetafileBuilder.rs:75`). It ran before
`code()` and wrote `IntermediateOutput::get_size()`
(`src/bundler/Chunk.rs:610`), which adds only the literal pieces. It
leaves out every path and HTML manifest that `code()` writes, and the
`//# debugId` and `//# sourceMappingURL` comments.

### Fix
- `Chunk::final_output_size` records the length of the emitted buffer.
Each place that makes the final buffer sets it: two in
`generateChunksInParallel.rs` (standalone HTML, in-memory) and one in
`writeOutputFilesToDisk.rs`.
- `metafile_builder::generate` builds each chunk fragment. Both callers
run it after `generate_chunks_in_parallel`, so the size is known.
esbuild defers this fragment in the same way.
- Removed: `get_size()`, `Chunk::metafile_chunk_json`, and the `unsafe
detach_lifetime()` in `generate`. The fragment was its only borrow that
was not `'static`.
- Verified: `test/bundler/metafile.test.ts`, 15 new cases, all fail on
1.4.3. Also the `esbuild/metafile`, `bundler_compile_splitting`,
`standalone`, `bundler_html` and `html-import-manifest` suites.

### Background
- The bundler prints a chunk before its final path exists, because the
path holds a content hash. A reference to another output is a
placeholder (a unique key) in the printed text.
- `IntermediateOutput::Pieces` holds the printed text, split at each
placeholder. `code()` joins the pieces and writes the final path at each
split.
- The bundler then appends the source map comment and writes the buffer
to disk, or returns it as a `BuildArtifact`.

<details><summary>Notes</summary>

Repro (1.4.3 and main at 5fce36e):

```sh
mkdir -p src
printf 'import { v1 } from "./m1.ts";\nimport { v2 } from "./m2.ts";\nconsole.log(v1 + v2);\n' > src/m0.ts
printf 'import c from "./c3.cjs";\nexport const v1 = "V1" + c.v3;\nconsole.log(v1);\n' > src/m1.ts
printf 'import c from "./c3.cjs";\nexport const v2 = "V2" + c.v3;\nconsole.log(v2);\n' > src/m2.ts
printf 'module.exports = { v3: "V3" };\n' > src/c3.cjs
bun build --splitting --target=bun --outdir=out --metafile=meta.json src/m0.ts src/m1.ts src/m2.ts
```

| output | on disk | `bytes` before | `bytes` after |
| --- | --- | --- | --- |
| `./m0.js` (2 imports) | 130 | 98 | 130 |
| `./m0-5begjve3.js` (1 import) | 195 | 179 | 195 |
| `./m1.js` (1 import) | 74 | 58 | 74 |
| `./m0-8x1hvvzq.js` (no import) | 1400 | 1400 | 1400 |

Each import costs 16 bytes, the length of `./m0-xxxxxxxx.js`. Other
cases measured on 1.4.3:

- `--sourcemap=linked`: `m0.js` is 207 on disk, the metafile said 98.
- `--sourcemap=inline`: 611 on disk, 98 in the metafile.
- `--sourcemap=external`: 176 on disk (the `//# debugId` comment, which
`code()` writes), 98 in the metafile.
- `import logo from "./logo.svg"` (file loader): 98 on disk, 79 in the
metafile.
- An HTML entry point: `index.html` is 218 on disk, 160 in the metafile.
- A server build that imports an HTML file (the manifest placeholder):
`server.js` now matches, 889 bytes.

esbuild 0.25.1 on a split build with source maps reports the exact file
size (`a.js`: 146 on disk, 146 in the metafile). Its linker passes
`len(outputContents)` to `jsonMetadataChunkCallback(finalOutputSize)`
after it appends the `sourceMappingURL` comment.

The new tests compare `outputs[key].bytes` with the size of each emitted
file:

- Code splitting with each `sourcemap` mode (`none`, `linked`, `inline`,
`external`), with an `outdir` (the disk loop) and without (the in-memory
loop).
- An asset referenced from JS, and a 128 KB asset referenced from CSS. A
stylesheet inlines a smaller file as a `data:` URL.
- An HTML entry point.
- A standalone HTML file (`compile` with `target: "browser"`). This is
the only case that reaches the third assignment. The test compares the
`bytes` of each inlined chunk with the text between its tags in the
document.
- The CLI with `--metafile` and `--metafile-md`. The test reads the
`[ENTRY: ... (N bytes)]` lines of the report.

With any one of the three assignments removed, the chunks of that path
report 0 and its cases fail.

More builds checked by hand against the fixed binary, all equal to the
file on disk: `--banner` and `--footer`, a shebang entry with
`--target=bun`, `--minify` with `--sourcemap=linked`, `--format=cjs
--bytecode`, `--public-path`, `--entry-naming
'[dir]/[name]-[hash].[ext]'` with entry points in subdirectories, and a
CSS entry point with an `@import` and a 200 KB `url()` asset.

Why a field on the chunk and not `OutputFile.size`: a standalone HTML
build gives each inlined chunk an empty placeholder `OutputFile` and
then filters those out of the list. The index of a chunk and the index
of its output file match only by a `debug_assert`, and not in that mode.

What changes for a user:

- `bytes` grows by the paths, by the `debugId` comment, and by the
`sourceMappingURL` comment. With `sourcemap: "inline"` that comment
holds the whole map. The number now equals `BuildArtifact.size`.
- The `--metafile-md` report uses the same numbers for "Total output
size", "Bundle size", the chunk sizes and the "% of Total" column.
- `--compile --target=browser` inlines the JS and CSS chunks into the
document. `index.html` now reports its real size (403, was 203). An
inlined chunk keeps its `outputs` entry, as before. Its `bytes` is the
size of the inlined code.
- `--compile` embeds each chunk in the executable. `bytes` is the size
of the embedded module.
- An error from `generate_chunk_json` now fails `generate` with the
existing "Failed to generate metafile" warning. Before,
`unwrap_or_default()` left an empty fragment and the JSON was not valid.
The only fallible call is a write to a `Vec`.

Not changed here, tracked in #42758:

- `.map`, `.jsc` and copied asset files have no `outputs` entry. This is
a different defect (a missing key, not a wrong value) and it adds keys
that users see.
- `inputs[..].bytesInOutput` counts a placeholder at its own length. A
fix needs a count per compile result after the substitution, as in
esbuild's `accurateFinalByteCount`.
- A standalone HTML build lists the inlined chunks as outputs.

Self-reviewed: 5 concerns raised, 5 addressed. A test for the standalone
HTML assignment. The `unsafe` that this change made needless. Stricter
test assertions (`statSync`, exact key lists, one combined CLI
assertion). The `debugId` wording in the commit message. A tracker for
the gaps above.

Open PRs that touch `MetafileBuilder.rs` but not this field: #38341,
#38624, #36974, #38666. None reads state that is gone when `generate`
runs. #38666 has a small text conflict in `Chunk.rs` where this PR
deletes `get_size()`.

</details>

<!-- robobun:evidence:begin -->

---

**no test proof** · iteration 0 · platform-specific test(s) that do not
run on this machine, deferring to CI, which covers all platforms:
test/bundler/metafile.test.ts

<!-- robobun:evidence:end -->
… in entry point order

The scan phase gives a file its source index when the parse result of
its importer arrives. Two builds of the same files number them in
different ways. Two places put that number into the output.

`MetafileBuilder::generate` listed `inputs` by source index. It now
walks `reachable_files`, the order that esbuild uses.

`compute_chunks` sorted the CSS chunks by their key, a hash of source
indices or of the entry point id. The chunks now keep the order in
which the entry points made them. The key stays as it is and only
tells two chunks apart.
@robobun robobun changed the title bundler: make --metafile output deterministic across build directories bundler: list metafile inputs in reachable-files order and CSS chunks in entry point order Oct 27, 2026
@robobun
robobun force-pushed the farm/e9ef9e0d/metafile-deterministic-inputs branch from 10b0f39 to 5a2cff7 Compare October 27, 2026 02:35
@robobun

robobun commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 9:09 PM PT - Oct 26th, 2026

✅ @robobun, your commit 5a2cff7c1d446b38a46fb44f693f9680c6a1eaeb passed in Build #133384! 🎉


🧪   To try this PR locally:

bunx bun-pr 36974

That installs a local version of the PR into your bun-36974 executable, so you can run:

bun-36974 --bun

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review completed

Nothing new to post: everything this review found is already covered by existing comments on this pull request or didn't merit a separate one.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants