Skip to content

bundler: write the metafile record of a JS output from the compile results of its chunk - #47503

Open
robobun wants to merge 3 commits into
robobun/feeff153/metafile-css-output-recordsfrom
robobun/bf7d20c5/metafile-js-output-inputs
Open

robobun wants to merge 3 commits into
robobun/feeff153/metafile-css-output-recordsfrom
robobun/bf7d20c5/metafile-js-output-inputs

Conversation

@robobun

@robobun robobun commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #47453.

Problem

  • The metafile record of a JS output omits a stylesheet whose class names the output prints. For import styles from "./a.module.css", outputs["./entry.js"].inputs is {"entry.js":{"bytesInOutput":35}}, and 48 of 112 bytes are in no record.
  • The JS print worker counts bytes in Chunk.files_with_parts_in_chunk (generateCompileResultForJSChunk.rs:145). A stylesheet has no key there (computeChunks.rs:308).
  • No user reported this. v1.3.6 to v1.4.4 have it.

Fix

Background

  • The metafile (--metafile) is esbuild's JSON build report. Each output lists its inputs with bytesInOutput.
  • Workers print a chunk in pieces. A compile result is one piece.
  • Weighed: a key for the stylesheet, but the linker reads the keys. Also a metafile-only patch, but the worker still drops the bytes.

Downsides

  • Behaviour change: such a JS output gains a record, also in --metafile-md.
  • Not as esbuild: a stylesheet that the output prints nothing of gets no 0-byte record (see Notes).
  • Cost: --metafile with 1,000 CSS modules takes +0.87% instructions. .text is equal.
Notes

Where this comes from

#47453 also says: "Not in a PR: the counter of the JS worker. [...] The metafile can add up the same number from the compile results, as it does for CSS here. I have that as a commit [...] A one-line gate on the metafile option in the worker is the other way. Nothing asks for either, so it is not in a PR."

The missing record now asks for the first of the two. The gate keeps the lookup that finds no key for a stylesheet, so it cannot write the record. With the sums taken from the compile results, nothing reads the counters. So they go, and a build with no metafile no longer pays for the metafile.

Question for a maintainer: the 0-byte record

esbuild writes "style.css": {"bytesInOutput": 0} in a JS output for a stylesheet that the output imports and prints nothing of. That is a bare import "./style.css", or a binding that nothing reads. Should Bun write it too?

  • With this PR, a stylesheet has a record only when the chunk has code of it.
  • Bun already writes a 0-byte record for each file of the chunk that prints nothing:
    • an empty file
    • a file that only re-exports
    • the page in the JS chunk of an HTML page
    • an asset that only a stylesheet references
  • The --compile --aot metafile writer (src/bundler/aot/metafile.rs) lists each input with no filter.
  • The bytes > 0 filter in chunk_inputs is not what leaves the record out. A bare import gives the chunk no compile result for the stylesheet. The filter acts in one case: a wrapped stylesheet whose wrapper the chunk drops. In that case esbuild keeps the wrapper and lists 139 bytes.
  • A "yes" can read js.files_in_chunk_order. For a bare import it is [a.module.css, bare.js] (from a run with a temporary print).
  • That list also has a stylesheet that an HTML page links and no JS imports: [page.css, plain.js, plain.html]. So the rule needs a check of which file imports the stylesheet.
  • A "yes" adds a record to each JS output with a bare CSS import. The three bun init React templates have one.

Records of the JS output (bun build --metafile, and esbuild 0.21.5 with --bundle --format=esm)

Build #47453 this PR esbuild
import styles from "./a.module.css", reads styles.foo entry.js 35 entry.js 35, a.module.css 48 a.module.css 36, entry.js 28
import { foo } from "./a.module.css" entry.js 18 entry.js 18, a.module.css 24 a.module.css 19, entry.js 18
import "./a.module.css" entry.js 22 entry.js 22 a.module.css 0, entry.js 22
import("./route.js") with no --splitting, where route.js has import "./x.css" route.js 68, entry.js 82 the same, and x.css 23 x.css 43, route.js 171, entry.js 95
--splitting: one.js, two.js and their shared shared.js read styles.foo one.js 50. shared.js 41. two.js 50 the same, and a.module.css 48 in each of the 3 outputs one.js 43. two.js 43. a.module.css 36, shared.js 34
import "./style.css", where style.css has url("./big.png") big.png 0, entry.js 22 the same style.css 0, entry.js 22
a bare import and a dead require() of a stylesheet of a package with no side effects entry.js 22 entry.js 22 a.module.css 139, own.css 0, entry.js 59

In the fourth row route.js is a wrapped file. So the chunk prints a wrapper for the stylesheet (var init_x = () => {} and a semicolon), and the stylesheet has a record. A bare import can give a record.

Size

100 components, and each one reads 20 class names of its own CSS module:

What stays as on #47453

These rules came with the counters of #25842. This PR does not decide them.

  • The order. The files of the chunk come first, in graph order. The stylesheets come after them, in the order the chunk first prints them. esbuild lists each source where the chunk first prints it.
  • A file of the chunk that prints nothing has a record with 0 bytes. Without --splitting, each live file that the entry point reaches is a file of the chunk. With --splitting, only a file with a compile result is one. So a file that only re-exports, or an empty file, keeps its 0-byte record. A file that only imports, or that tree shaking leaves nothing of, has none.
  • An asset that only a stylesheet references is a file of the JS chunk, with 0 bytes. bundler: keep assets referenced by stylesheets and HTML documents out of JS chunks #38337 removes that record.
  • bytesInOutput counts the placeholder of an asset path, not the final path. esbuild counts the final path.
  • An HTML output has inputs: {}.

Weighed and not done: one rule for the three kinds of chunk, as esbuild has. That rule adds up the compile results in print order. It moves the records of today.

Bugs in what a JS chunk prints for a stylesheet

They are not in this PR. The metafile counts that code as it is, and seven of the new tests build such code.

No issue holds these bugs. The three PRs hold them. They are open since August.

Deleted

  • The counter block of the JS print worker, and the AtomicUsize values of the map.
  • The // TODO: metafile marker in the join loop of postProcessJSChunk.rs. esbuild collects this list there. chunk_inputs reads the same compile results.

Numbers (release builds with LTO, of #47453 at 37777f6 and of this PR)

  • JS print worker, 2,001 modules, no metafile: 2,001 -> 0 map lookups, 2,001 -> 0 lock add (gdb hit counts). The worker of bundler: write the metafile record of a CSS output from the compile results of its chunk #47453 is the one of main. objdump shows no lock instruction in the worker of this PR.
  • The same build with --metafile: chunk_inputs does the 2,001 lookups.
  • compute_chunks, 2,001 modules: 138 -> 119 allocator calls (-32,736 bytes asked for). metafile_builder::generate with --metafile: 42,052 calls in both for 2,001 modules, and 88 -> 98 for the example with one stylesheet (gdb).
  • Release binary: .text 108,799,721 bytes and .rodata 26,448,844 bytes in both. JS print worker 2,112 -> 1,718 bytes (427 -> 328 instructions). metafile_builder::generate 10,873 -> 12,081 bytes. compute_chunks 45,711 -> 45,598 bytes.
  • files_with_parts_in_chunk: 16 -> 8 bytes per key. sizeof(Chunk) 792 -> 792.
  • User instructions, median of 21 interleaved runs, 2,001 TS modules: no metafile -0.14% (179,881,823 -> 179,635,048), with --metafile -0.04% (194,912,374 -> 194,837,670). The spread of the runs is 0.22% to 0.37%, so both are inside it.
  • User instructions, 1,000 CSS modules: no metafile -0.08% (151,501,235 -> 151,381,551, inside the spread), with --metafile +0.87% (160,617,099 -> 162,017,040). That build writes 1,000 new records.
  • User instructions, 250 entry points that each bundle the same 4,000 modules, no metafile: -0.77% (11,649,344,099 -> 11,560,219,903, median of 11, spread 0.19% to 0.41%). That build prints 1,000,500 pieces, and each piece did a lookup and a lock add.
  • Wall time of that build, 15 interleaved rounds with 12 threads: median 0.549 s -> 0.502 s. The ranges overlap (0.463 to 0.653 s, and 0.428 to 0.557 s). This PR is faster in 13 of the 15.

How the numbers were taken:

  • Instructions are user instructions of the process (perf_event_open), with 2 pool threads.
  • Allocator calls are gdb counts of the public mimalloc entries on the bundle thread. The sizes that go are those of the counter column: 32 bytes, then 64 to 16,384.
  • sizeof(Chunk) is the size of the chunk array of a build with one chunk.
  • Sizes are from size -A and nm -S.
  • perf, valgrind, strace and bloaty are not on the machine.

Records that change

23 builds with both binaries. 16 have no stylesheet code in a JS chunk: JS only, --splitting, --format=cjs, --minify, stylesheet entry points, --css-chunking, HTML pages. In 7 a JS chunk prints code of a stylesheet.

Tests

20 of the 25 new tests fail on #47453:

  • the six ways to use a stylesheet from JS, each in ESM and CJS output (12)
  • two pieces of one stylesheet
  • the order of two stylesheets, with and without --splitting (2)
  • the wrapper of a bare import
  • a copy in each chunk, with and without --splitting (2)
  • the JS chunk of an HTML page
  • the CLI, with --metafile-md

5 pass on #47453 and pin what must stay:

  • no record for a bare import, and for a binding that nothing reads (2)
  • no record for a wrapper that the chunk drops. This one fails without the bytes > 0 filter.
  • the 0-byte record of a file of the chunk, with and without --splitting (2)

Suites run with the debug build

metafile 104 pass. All pass: esbuild/metafile 12, aot/cli/metafile 4, css/css-modules 46, esbuild/css 68, esbuild/splitting 26, bundler_html 26, html-import-manifest 10, bundler_compile_splitting 30, bundler_loader 62, bundler_bun 22, bundler_naming 27, bundler_splitting 209, esbuild/default 155, bundler_plugin 68, bundler_html_server 4, bake/dev-and-prod 12.

Self-review

17 concerns. 16 addressed:

  • In the code: the comment on the filter named a difference from esbuild that the filter does not cause. The doc line of the map now says that a chunk prints code that has no key.
  • In the tests: the order of the stylesheet records, the wrapper of a bare import, and the 0-byte rule with --splitting.
  • In this text:

Not done (1): a new tracker for those bugs. The three open PRs hold them, and #42758 was closed because no user hit what it listed.

Other open PRs on these lines (trial merges)


no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/bundler/metafile.test.ts

…ults of its chunk

A JS chunk prints the class names of a stylesheet that it uses. The
stylesheet is not a file of the chunk, so the byte counter of the print
worker had no key for it, and the metafile record of the JS output did
not list it.

The metafile now adds up the compile results of the chunk. The files of
the chunk keep their place and their order. A source that is not one of
them gets a record after them when the chunk has code of it.

The print worker no longer counts, so files_with_parts_in_chunk is a set
again.
… the order of the stylesheet records

The comment on the filter named a difference from esbuild that the filter
does not cause. The filter is for a wrapped stylesheet whose wrapper the
chunk drops.

New tests: the order of two stylesheet records, the wrapper that the chunk
prints for a bare import in a wrapped file, and the 0-byte record of a file
of the chunk with and without --splitting.
@robobun

robobun commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 5:18 PM PT - Oct 30th, 2026

❌ @robobun, your commit e2ea6f9 has 1 failures in Build #135924 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 47503

That installs a local version of the PR into your bun-47503 executable, so you can run:

bun-47503 --bun

@robobun

robobun commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: ready for review. The diff is green in CI.

Reproduced on the base, #47453 at 37777f6, and on main: entry.js has import styles from "./a.module.css" and reads styles.foo. bun build ./entry.js --outdir=out --metafile=meta.json prints the class names in out/entry.js (48 of 112 bytes). outputs["./entry.js"].inputs lists only entry.js. esbuild lists the stylesheet there.

20 of the 25 new tests in test/bundler/metafile.test.ts fail on the base and pass on this branch. The other 5 are pins and pass on both. In CI the file passes on each platform: 104 passes on Linux and macOS, and 103 on Windows with the one skip that the file had before.

The one red test of the CI run is test/aot/engine/ir-6.test.ts. It is also red on main (build 135919), and this diff does not touch it. The other failures passed on retry.

This PR is stacked on #47453.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline finding, I checked the two behaviors the deleted worker counter had that the new chunk_inputs no longer spells out: the runtime source is still dropped from the record by the existing Index::RUNTIME filter in generate_chunk_json (MetafileBuilder.rs:130), and a PrintResult::Err still counts as 0 bytes because CompileResult::code() returns an empty slice for it. The remaining readers of files_with_parts_in_chunk (findAllImportedPartsInJSOrder, computeCrossChunkDependencies, generateChunksInParallel) only use keys/contains/count, so the () value type affects none of them.

Extended reasoning...

The change replaces the atomic per-source byte counters in the JS codegen workers with a post-codegen sum over each JS chunk's compile results in MetafileBuilder.rs, turns files_with_parts_in_chunk into a plain set, updates the Send/Sync and prologue CONCURRENCY comments accordingly, and adds ~340 lines of metafile tests. It touches no security-sensitive surface. A confirmed inline finding on the bytes > 0 filter and the resulting esbuild divergence (no 0-byte stylesheet record for a bare CSS import) is a product decision the author explicitly asks a maintainer to settle, so a human look is needed.

@robobun

robobun commented Oct 31, 2026

Copy link
Copy Markdown
Collaborator Author

Reply to the review: its one finding is the 0-byte record for a stylesheet that the JS output prints nothing of. That is the open question in the Notes of the description, so the code stays as it is until a maintainer answers. I ran the suggested source, js.files_in_chunk_order. The result is in the Notes and in the thread on MetafileBuilder.rs:80: the list has the stylesheet of a bare import, and it also has a stylesheet that an HTML page links and no JS imports.

@robobun

robobun commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator Author

A note on the merge order with #47624. That PR makes Content::Html a struct variant (Content::Html { js_chunk_index }). This branch merges with it with no textual conflict, but two patterns in MetafileBuilder.rs then fail to compile: ChunkContent::Html => Ok(Vec::new()) and ChunkContent::Css(_) | ChunkContent::Html => false. They come from #47453, which has the same note.

ChunkContent::Html { .. } compiles on main today and after #47624, so the edit can go in now.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant