Skip to content

bundler: list tree-shaken files in the metafile inputs so import edges resolve - #38624

Open
robobun wants to merge 2 commits into
mainfrom
farm/b572f9a9/metafile-inputs-reachable-files
Open

robobun wants to merge 2 commits into
mainfrom
farm/b572f9a9/metafile-inputs-reachable-files

Conversation

@robobun

@robobun robobun commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Problem

  • With --metafile, a file that tree shaking removes entirely is missing from inputs, while the imports[] of the file that imports it still points at it: the edge has no target and is not marked external. Consumers that walk inputs edges (esbuild's analyzer, --metafile-md) hit a dangling edge. Reproduces on bun 1.4.0 and main.
  • Two ways to get there, repro in the details block below:
    • import { x } from "pkg" where pkg has "sideEffects": false and x is unused: inputs has only entry.js, whose imports list node_modules/pkg/index.js.
    • function unused() { return require("./lazy.js") } (no sideEffects field anywhere): lazy.js and the effect.js it imports are both missing, entry.js still lists lazy.js.
  • esbuild lists these files in inputs (with their own imports) and leaves them out of outputs[].inputs, so the edges resolve. bun build itself already counts them ("Bundled 2 modules" for the first repro).
  • Cause: generate in src/bundler/linker_context/MetafileBuilder.rs seeds the set of inputs to write from chunk.files_with_parts_in_chunk of every chunk (lines 223-230 on main). That map is filled in src/bundler/linker_context/computeChunks.rs:291-292 from the reachable files that are also in files_live, so it only holds files that survived tree shaking. The per-input imports are written from the importer's import records (MetafileBuilder.rs:299-312), whose source_index still points at the removed file.

Fix

  • generate seeds the inputs from c.graph.reachable_files instead of chunk membership. Nothing else changes: the per-input fields, the index-order emission, and outputs[].inputs (still derived from files_with_parts_in_chunk) are as before.
  • Why this is the right set: reachable_files is the closure of the entry points over import records with a valid source_index, computed before tree shaking (find_reachable_files, src/bundler/bundle_v2.rs). The import records written into imports[] are exactly the edges that traversal followed, so every bundled import edge now lands on an inputs key. Conversely every file that was in a chunk is reachable (the chunk maps are filled from reachable_files, see above), so nothing that was listed before disappears. It is the set esbuild writes as well (checked against esbuild 0.25.12 on both repros) and the one the "Bundled N modules" summary counts (reachable_files_count in bundle_v2.rs).
  • outputs[].inputs is deliberately left chunk-based: a removed file contributed nothing to any output, and esbuild omits it there too (covered by the tests).
  • Verified with test/bundler/metafile.test.ts: metafile lists a sideEffects: false module that tree shaking dropped from the output, metafile lists modules only reached from tree-shaken code, and what they import (also checks the removed module's own import edge and that outputs are unchanged), and markdown lists a module that tree shaking dropped from the output (--metafile-md module list and reverse dependency). All three fail on the released build (USE_SYSTEM_BUN=1, the only diff being the missing entries) and pass with bun bd test; the rest of the file (47 tests), test/bundler/esbuild/metafile.test.ts and css/MetafileCSSBundleTwoToOne pass with the debug build. cargo clippy -p bun_bundler --no-deps and cargo fmt --check are clean.
  • Side effect worth knowing: a CSS entry point's stylesheets (which have no JS chunk and an empty CSS chunk map today) now show up in inputs as well; the outputs half of that is bundler: read the metafile size of a copied input from its asset file #38341 (its source merges cleanly with this; the test file conflicts only on adjacent import lines). bundler: keep assets referenced by stylesheets and HTML documents out of JS chunks #38337 adds document assets to the same seeding loop; those assets are reachable, so this change covers its metafile part and that hunk can be dropped when the two meet. bundler: report a split dynamic import as its input file in the metafile #38465 builds a map inside the loop this removes and needs a small rebase. bundler: list metafile inputs in reachable-files order and CSS chunks in entry point order #36974 (input ordering) merges cleanly.

Background

  • Metafile: esbuild's build report format. inputs maps each source file to its size and the imports found in it (edges between source files, or external: true); outputs maps each emitted file to the inputs it was built from (bytesInOutput) and the chunks it imports. Bun writes outputs per chunk during chunk generation and assembles inputs afterwards in metafile_builder::generate.
  • Reachable files: before linking, the bundler walks import records from the entry points and records every file it reaches (graph.reachable_files). This is the set of files that were parsed into the build.
  • Tree shaking: the linker then marks files and parts live starting from the entry points. A file with no used exports whose package declares "sideEffects": false, or one only referenced from code that is itself dead, is never marked live. compute_chunks places only live files into chunks, and a chunk's files_with_parts_in_chunk is what outputs[].inputs and, until this change, inputs were generated from.
Repro
mkdir -p d/node_modules/pkg && cd d
printf 'import { x } from "pkg";\nconsole.log("hi");\n' > entry.js
printf '{"name":"pkg","main":"index.js","sideEffects":false}\n' > node_modules/pkg/package.json
printf 'export const x = 1;\n' > node_modules/pkg/index.js
bun build entry.js --outdir out --metafile=meta.json

Before (bun 1.4.0 and main), inputs:

{
  "entry.js": {
    "bytes": 44,
    "imports": [{ "path": "node_modules/pkg/index.js", "kind": "import-statement", "original": "pkg" }],
    "format": "esm"
  }
}

After (same as esbuild's inputs for this project):

{
  "entry.js": {
    "bytes": 44,
    "imports": [{ "path": "node_modules/pkg/index.js", "kind": "import-statement", "original": "pkg" }],
    "format": "esm"
  },
  "node_modules/pkg/index.js": { "bytes": 20, "imports": [], "format": "esm" }
}

outputs["./entry.js"].inputs is { "entry.js": { "bytesInOutput": 19 } } before and after.

Second shape, without any sideEffects field:

printf 'function unused() { return require("./lazy.js"); }\nconsole.log("hi");\n' > entry.js
printf 'import "./effect.js";\nmodule.exports = 1;\n' > lazy.js
printf 'console.log("effect");\n' > effect.js
bun build entry.js --outdir out --metafile=meta.json

Before, inputs keys: ["entry.js"], with entry.js importing lazy.js. After: ["entry.js", "lazy.js", "effect.js"], with lazy.js importing effect.js. esbuild lists the same three.

--metafile-md for the first repro, before: | Input modules | 1 | and an [IMPORT: entry.js -> node_modules/pkg/index.js] line with no matching [MODULE: ...] or [IMPORTED_BY: ...]. After: | Input modules | 2 |, [MODULE: node_modules/pkg/index.js], [IMPORTED_BY: node_modules/pkg/index.js <- entry.js].

Also checked with the fixed build: an HTML entry point (scripts, stylesheet, <img>, url() asset), a JS entry importing CSS, a sideEffects: false barrel with an unused re-export, and the first repro under --splitting all produce the same inputs as before plus, where applicable, the removed modules; a CSS entry point now lists its stylesheets (see the last Fix bullet).

…iles left in a chunk

The inputs section was seeded from chunk membership, so a file that tree
shaking removed entirely (a sideEffects: false module with no used exports,
or a module only reached from dead code) was missing from inputs while its
importer's imports[] still pointed at it. Seed it from the linker's
reachable files instead, which is also what esbuild lists; outputs[].inputs
stays chunk-based.
@robobun

robobun commented Aug 14, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: reproduced on bun 1.4.0 and on a debug build of main with the two repros in the description (sideEffects: false package with an unused import, and a module only reached from a dead require()). Fix and tests are in this PR; the three new tests in test/bundler/metafile.test.ts fail on the released build and pass with the fix.

CI (build 96353): every lane that ran passed (177 jobs). The only red is darwin 14 aarch64 - test-bun, whose two shards expired three times without ever being picked up by an agent, so nothing ran or failed there; the remaining annotations are tests that passed on retry and are unrelated to the metafile. Ready for review.

@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 18 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: f2a6f434-1a05-4d0e-aab9-b6a7a096e195

📥 Commits

Reviewing files that changed from the base of the PR and between 2f5c180 and 3d2582e.

📒 Files selected for processing (2)
  • src/bundler/linker_context/MetafileBuilder.rs
  • test/bundler/metafile.test.ts

Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — focused fix that seeds metafile inputs from reachable_files instead of chunk membership, matching esbuild.

What was reviewed:

  • Verified the superset claim: files_with_parts_in_chunk is filled from reachable_files filtered by files_live (computeChunks.rs:291-292), so nothing previously listed disappears.
  • Checked the dropped bounds guard is safe: find_reachable_files marks indices via a bitset sized to input_files.len(), so every reachable index is < sources.len().
  • New tests assert exact inputs shape for both tree-shaking paths and confirm outputs[].inputs stays chunk-based; Windows path-separator concern ruled out (pretty paths use forward slashes here, same as neighboring --metafile-md tests).
Extended reasoning...

Overview

The PR changes one loop in src/bundler/linker_context/MetafileBuilder.rs: the seen_sources bitset that decides which files appear as top-level inputs keys in the metafile is now seeded from c.graph.reachable_files instead of the union of chunk.files_with_parts_in_chunk across all chunks. Three new tests in test/bundler/metafile.test.ts cover a sideEffects: false package dropped by tree shaking, a module reached only from dead code (and its transitive import), and the --metafile-md rendering of the same. The import statement for bunEnv/bunExe is hoisted to the existing top-level harness import.

Security risks

None. The metafile is diagnostic JSON describing what the bundler parsed; no auth, no untrusted input parsing, no filesystem writes beyond what bun build already does.

Level of scrutiny

Low-to-medium. The Rust change is a net -3 lines swapping one iteration source for another in report-generation code — not on any hot path or affecting bundle output. I traced the two invariants that make the swap safe: (1) reachable_files is a superset of what chunks contain, since computeChunks.rs:291-292 fills chunk membership by iterating reachable_files and filtering on files_live, so no previously-listed input can disappear; (2) every index in reachable_files is < sources.len() because find_reachable_files (bundle_v2.rs:1952) uses a visited bitset sized to input_files.len(), so the removed < sources.len() guard was redundant and seen_sources.set() cannot go out of bounds. The runtime index is still filtered by the existing Index::RUNTIME check in the emission loop.

Other factors

The tests follow the file's conventions exactly: tempDir + using, bunEnv/bunExe, concurrent stdout/stderr/exit drain, test.concurrent for the subprocess-spawning cases, and toEqual on the full inputs object (strong assertion). They also assert the negative — outputs[].inputs still lists only entry.js — which pins down that the fix is scoped to the top-level inputs. The PR description confirms USE_SYSTEM_BUN=1 fails and bun bd test passes on all three, plus the rest of the file and esbuild/metafile.test.ts. The disclosed side effect (CSS entry-point stylesheets now appear in inputs) is a strict improvement in the same direction and is called out with the related PR numbers. No prior human review comments to address.

Jarred-Sumner pushed a commit that referenced this pull request Sep 15, 2026
…].bytes (#42760)

### Problem
- In the JSON metafile (`bun build --metafile`, `Bun.build({ metafile:
true })`), `outputs[path].bytes` is smaller than the emitted file when
the chunk references another output. On 1.4.3 `m0.js` is 130 bytes on
disk, 98 in the metafile. `bun.d.ts` documents the field as "Size of the
output file in bytes".
- The cause is `generate_chunk_json`
(`src/bundler/linker_context/MetafileBuilder.rs:75`). It ran before
`code()` and wrote `IntermediateOutput::get_size()`
(`src/bundler/Chunk.rs:610`), which adds only the literal pieces. It
leaves out every path and HTML manifest that `code()` writes, and the
`//# debugId` and `//# sourceMappingURL` comments.

### Fix
- `Chunk::final_output_size` records the length of the emitted buffer.
Each place that makes the final buffer sets it: two in
`generateChunksInParallel.rs` (standalone HTML, in-memory) and one in
`writeOutputFilesToDisk.rs`.
- `metafile_builder::generate` builds each chunk fragment. Both callers
run it after `generate_chunks_in_parallel`, so the size is known.
esbuild defers this fragment in the same way.
- Removed: `get_size()`, `Chunk::metafile_chunk_json`, and the `unsafe
detach_lifetime()` in `generate`. The fragment was its only borrow that
was not `'static`.
- Verified: `test/bundler/metafile.test.ts`, 15 new cases, all fail on
1.4.3. Also the `esbuild/metafile`, `bundler_compile_splitting`,
`standalone`, `bundler_html` and `html-import-manifest` suites.

### Background
- The bundler prints a chunk before its final path exists, because the
path holds a content hash. A reference to another output is a
placeholder (a unique key) in the printed text.
- `IntermediateOutput::Pieces` holds the printed text, split at each
placeholder. `code()` joins the pieces and writes the final path at each
split.
- The bundler then appends the source map comment and writes the buffer
to disk, or returns it as a `BuildArtifact`.

<details><summary>Notes</summary>

Repro (1.4.3 and main at 5fce36e):

```sh
mkdir -p src
printf 'import { v1 } from "./m1.ts";\nimport { v2 } from "./m2.ts";\nconsole.log(v1 + v2);\n' > src/m0.ts
printf 'import c from "./c3.cjs";\nexport const v1 = "V1" + c.v3;\nconsole.log(v1);\n' > src/m1.ts
printf 'import c from "./c3.cjs";\nexport const v2 = "V2" + c.v3;\nconsole.log(v2);\n' > src/m2.ts
printf 'module.exports = { v3: "V3" };\n' > src/c3.cjs
bun build --splitting --target=bun --outdir=out --metafile=meta.json src/m0.ts src/m1.ts src/m2.ts
```

| output | on disk | `bytes` before | `bytes` after |
| --- | --- | --- | --- |
| `./m0.js` (2 imports) | 130 | 98 | 130 |
| `./m0-5begjve3.js` (1 import) | 195 | 179 | 195 |
| `./m1.js` (1 import) | 74 | 58 | 74 |
| `./m0-8x1hvvzq.js` (no import) | 1400 | 1400 | 1400 |

Each import costs 16 bytes, the length of `./m0-xxxxxxxx.js`. Other
cases measured on 1.4.3:

- `--sourcemap=linked`: `m0.js` is 207 on disk, the metafile said 98.
- `--sourcemap=inline`: 611 on disk, 98 in the metafile.
- `--sourcemap=external`: 176 on disk (the `//# debugId` comment, which
`code()` writes), 98 in the metafile.
- `import logo from "./logo.svg"` (file loader): 98 on disk, 79 in the
metafile.
- An HTML entry point: `index.html` is 218 on disk, 160 in the metafile.
- A server build that imports an HTML file (the manifest placeholder):
`server.js` now matches, 889 bytes.

esbuild 0.25.1 on a split build with source maps reports the exact file
size (`a.js`: 146 on disk, 146 in the metafile). Its linker passes
`len(outputContents)` to `jsonMetadataChunkCallback(finalOutputSize)`
after it appends the `sourceMappingURL` comment.

The new tests compare `outputs[key].bytes` with the size of each emitted
file:

- Code splitting with each `sourcemap` mode (`none`, `linked`, `inline`,
`external`), with an `outdir` (the disk loop) and without (the in-memory
loop).
- An asset referenced from JS, and a 128 KB asset referenced from CSS. A
stylesheet inlines a smaller file as a `data:` URL.
- An HTML entry point.
- A standalone HTML file (`compile` with `target: "browser"`). This is
the only case that reaches the third assignment. The test compares the
`bytes` of each inlined chunk with the text between its tags in the
document.
- The CLI with `--metafile` and `--metafile-md`. The test reads the
`[ENTRY: ... (N bytes)]` lines of the report.

With any one of the three assignments removed, the chunks of that path
report 0 and its cases fail.

More builds checked by hand against the fixed binary, all equal to the
file on disk: `--banner` and `--footer`, a shebang entry with
`--target=bun`, `--minify` with `--sourcemap=linked`, `--format=cjs
--bytecode`, `--public-path`, `--entry-naming
'[dir]/[name]-[hash].[ext]'` with entry points in subdirectories, and a
CSS entry point with an `@import` and a 200 KB `url()` asset.

Why a field on the chunk and not `OutputFile.size`: a standalone HTML
build gives each inlined chunk an empty placeholder `OutputFile` and
then filters those out of the list. The index of a chunk and the index
of its output file match only by a `debug_assert`, and not in that mode.

What changes for a user:

- `bytes` grows by the paths, by the `debugId` comment, and by the
`sourceMappingURL` comment. With `sourcemap: "inline"` that comment
holds the whole map. The number now equals `BuildArtifact.size`.
- The `--metafile-md` report uses the same numbers for "Total output
size", "Bundle size", the chunk sizes and the "% of Total" column.
- `--compile --target=browser` inlines the JS and CSS chunks into the
document. `index.html` now reports its real size (403, was 203). An
inlined chunk keeps its `outputs` entry, as before. Its `bytes` is the
size of the inlined code.
- `--compile` embeds each chunk in the executable. `bytes` is the size
of the embedded module.
- An error from `generate_chunk_json` now fails `generate` with the
existing "Failed to generate metafile" warning. Before,
`unwrap_or_default()` left an empty fragment and the JSON was not valid.
The only fallible call is a write to a `Vec`.

Not changed here, tracked in #42758:

- `.map`, `.jsc` and copied asset files have no `outputs` entry. This is
a different defect (a missing key, not a wrong value) and it adds keys
that users see.
- `inputs[..].bytesInOutput` counts a placeholder at its own length. A
fix needs a count per compile result after the substitution, as in
esbuild's `accurateFinalByteCount`.
- A standalone HTML build lists the inlined chunks as outputs.

Self-reviewed: 5 concerns raised, 5 addressed. A test for the standalone
HTML assignment. The `unsafe` that this change made needless. Stricter
test assertions (`statSync`, exact key lists, one combined CLI
assertion). The `debugId` wording in the commit message. A tracker for
the gaps above.

Open PRs that touch `MetafileBuilder.rs` but not this field: #38341,
#38624, #36974, #38666. None reads state that is gone when `generate`
runs. #38666 has a small text conflict in `Chunk.rs` where this PR
deletes `get_size()`.

</details>

<!-- robobun:evidence:begin -->

---

**no test proof** · iteration 0 · platform-specific test(s) that do not
run on this machine, deferring to CI, which covers all platforms:
test/bundler/metafile.test.ts

<!-- robobun:evidence:end -->

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant