build: re-run glob-driven codegen steps when one of their inputs is deleted

[robobun]

Problem

• On a warm tree, deleting a file that a codegen step globs does not re-run that step. rm src/js/internal/foo.ts && bun bd leaves InternalModuleRegistry+enum.h (and the module's code) in the binary until some surviving input of the same step is edited. Reproduced on linux-x64 with a probe module: after removing it, ninja -d explain codegen/InternalModuleRegistry+enum.h reports no work to do and the header still contains InternalZzProbe (transcript below).
• Cause: the edges in scripts/build/codegen.ts list the globbed files as inputs and nothing carries the input set. Ninja re-runs an edge when an input is newer than its outputs or the command line changed; after a deletion, configure regenerates build.ninja with a shorter input list, the command is unchanged, and the outputs are newer than every remaining input, so the edge is up to date.
• The same shape is in every step that decides what to generate by looking at the tree itself or by bundling, with the list absent from its command line: emitJsModules (bundle-modules readdirs src/js), emitBindgen (bindgen.ts scans for .bind.ts), emitHostExports (scrapes src/runtime + src/jsc .rs files), emitBakeCodegen and emitBunError (bundles; which files exist decides how imports resolve). emitCppBind already handled it by writing codegen/cxx-sources.txt at configure time and listing it as an implicit input.
• Not affected: emitGeneratedClasses, emitBindgenV2 and emitNodeFallbacks pass the file list on the command line, so a deletion changes the command and ninja re-runs them ("command line changed"); checked with a synthetic ninja file. The cargo edge is also left alone: a .rs deletion that matters comes with an edit to the file that declared the module, and cargo's own fingerprinting decides what recompiles.

Fix

• codegen.ts: the cxx-sources.txt logic becomes sourceListFile(cfg, name, files), which writes codegen/<name>-sources.txt with writeIfChanged and returns the path. emitCppBind uses it (same path and content as before, so existing trees do not re-run cppbind), and the five steps above list their manifest as an implicit input: js-sources.txt, bindgen-sources.txt, host-exports-sources.txt, bake-sources.txt, bun-error-sources.txt.
• Why this is correct: the manifest is the input set as a file, so a deletion becomes something ninja can see (a rewritten manifest, newer than the outputs), while an unchanged set leaves the mtime alone and a reconfigure stays a no-op. It is written at configure time, which is the only point where the glob is re-evaluated anyway (adding a file already needs the reconfigure that bun bd always does). The codegen rule has restat = 1 and the scripts use writeIfNotChanged, so a set change whose output is unchanged re-runs the step and nothing downstream, as with cxx-sources.txt today.
• Cost: trees configured before this change re-run the five steps once when their manifests first appear, like any new input.
• Verified:
  • test/internal/source-lints/build-codegen-source-lists.test.ts (the source-lints dir per its README: build-script unit test, no bun binary involved): emits the six steps into a temp build dir with made-up source lists and checks that each edge has its manifest as an implicit input listing exactly the files the edge is fed (for host-exports, only the .rs files in the scrape scope), that an unchanged reconfigure rewrites none of them (content and mtime), and that dropping one JS module rewrites only js-sources.txt. Passes with bun bd test; fails with scripts/ stashed, and also fails (js-sources.txt missing) with only the manifest wiring removed and the exports kept. To make the steps reachable without emitCodegen() (which spawns bun for bindgenv2's list-outputs, several seconds per call under a debug build), Ctx and the six emitters are exported.
  • Real ninja, linux-x64 debug tree: removing the probe module now re-runs bundle-modules in the same bun bd (explain names js-sources.txt) and the module is gone; a reconfigure with no changes is still no work to do; removing a .bind.ts dirties the bindgen edge the same way (transcripts below).
  • bunx tsc --noEmit -p scripts/build/tsconfig.json reports no errors in codegen.ts; the rest of test/internal/source-lints/ and the existing test/internal/build-* tests still pass under bun bd test.
  • CI: every build lane (linux, musl, android, darwin, windows, freebsd) configures and builds with the new manifests, and the binary-size annotation shows every binary at +0.0 KB against the main canary, as expected for a change that only affects when codegen re-runs.
• scripts/build/CLAUDE.md: the "Add a codegen step" entry says when a step needs a manifest.

Background

• Ninja dirtiness: before running, ninja stats every declared input of an edge and re-runs the edge if any input is newer than the oldest output, if an output is missing, or if the edge's command differs from the one recorded in .ninja_log. The declared inputs come from build.ninja, which configure regenerates from a fresh glob on every bun bd; a file that is no longer in the list is simply not stat'd, so its removal is invisible unless something else records that the list changed.
• Implicit input (| file on a build line): tracked for dirtiness exactly like an explicit input, but not passed to the command as $in. The codegen and esbuild rules do not use $in at all, which is why the list itself is not on the command line and why the manifest goes in this slot.
• writeIfChanged (scripts/build/fs.ts): configure-time write that skips the write when the content is identical, so the file's mtime only moves when the content does. restat = 1 on the codegen rule: after the command runs, ninja re-stats the outputs and prunes downstream edges whose inputs did not actually change, and records the newest input mtime so an input that triggered a no-op run does not trigger it again.

Two neighbouring gaps found on the way are filed separately and not changed here: the bundle-modules edge does not list src/jsc/modules/NativeModuleList.h or src/js/builtins/BunBuiltinNames.h, which its scripts read, and the PCH picking up a regenerated header one build late is #37992.

Probe transcripts (linux-x64, build/debug, target = codegen/InternalModuleRegistry+enum.h)

Unfixed:

$ echo 'export default {};' > src/js/internal/zz_probe.ts
$ bun scripts/build.ts --profile=debug --configure-only && ninja -C build/debug codegen/InternalModuleRegistry+enum.h
[1/1] gen JS modules (bundle-modules)
$ grep -c ZzProbe build/debug/codegen/InternalModuleRegistry+enum.h
1
$ rm src/js/internal/zz_probe.ts
$ bun scripts/build.ts --profile=debug --configure-only && grep -c zz_probe build/debug/build.ninja
0
$ ninja -C build/debug -d explain codegen/InternalModuleRegistry+enum.h
ninja: no work to do.
$ grep -c ZzProbe build/debug/codegen/InternalModuleRegistry+enum.h
1

Fixed (same sequence):

$ rm src/js/internal/zz_probe.ts
$ bun scripts/build.ts --profile=debug --configure-only && ninja -C build/debug -d explain codegen/InternalModuleRegistry+enum.h
ninja explain: recorded mtime of codegen/WebCoreJSBuiltins.cpp older than most recent input codegen/js-sources.txt (...)
[1/1] gen JS modules (bundle-modules)
$ grep -c ZzProbe build/debug/codegen/InternalModuleRegistry+enum.h
0
$ bun scripts/build.ts --profile=debug --configure-only && ninja -C build/debug codegen/InternalModuleRegistry+enum.h
ninja: no work to do.

Fixed, bindgen (dry run after moving src/jsc/bindgen_test.bind.ts away and reconfiguring):

$ ninja -C build/debug -n -d explain codegen/GeneratedBindings.cpp
ninja explain: recorded mtime of codegen/GeneratedBindings.cpp older than most recent input codegen/bindgen-sources.txt (...)
[1/1] gen .bind.ts → GeneratedBindings.cpp

Manifests written on this tree after the change (cxx-sources.txt kept its old mtime, the content is unchanged):

bake-sources.txt          24 lines
bindgen-sources.txt        7 lines
bun-error-sources.txt     13 lines
cxx-sources.txt          596 lines
host-exports-sources.txt 533 lines
js-sources.txt           222 lines

Synthetic check that a list carried on the command line already re-runs (the generate-classes / bindgenv2 / build-fallbacks shape): with command = ls $in > /dev/null && echo out > $out, dropping an input gives ninja explain: command line changed for out; with the list not in the command, no work to do.

---

[stamp-90s] gate passed · iteration 1 · 3 files touched

passes on PR (with fix)

Test-only change.

Debug/ASAN (expected pass):
$ bun bd test 'test/internal/source-lints/build-codegen-source-lists.test.ts'
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test test/internal/source-lints/build-codegen-source-lists.test.ts
bun test v1.4.0 (085f257e8)

test/internal/source-lints/build-codegen-source-lists.test.ts:
(pass) codegen steps that glob their inputs track the set of inputs > each step's edge has a manifest as an implicit input, listing the files the edge is fed [907.60ms]
(pass) codegen steps that glob their inputs track the set of inputs > a manifest is rewritten exactly when its list changes [425.69ms]

 2 pass
 0 fail
 5 expect() calls
Ran 2 tests across 1 file. [7.52s]
Exit: 0

diff hotspot

scripts/build/CLAUDE.md                            |   2 +-
 scripts/build/codegen.ts                           |  88 +++++--
 .../build-codegen-source-lists.test.ts             | 279 +++++++++++++++++++++
 3 files changed, 343 insertions(+), 26 deletions(-)

gate history · 2 passed · 0 rejected · iteration 1

evidence per changed file

file                                                      reads  edits  tests
scripts/build/CLAUDE.md                                       2      2      0
scripts/build/codegen.ts                                      4     20      0
…nternal/source-lints/build-codegen-source-lists.test.ts      1      0      0

[coderabbitai]

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 13 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: fa3332e8-e7f6-490e-b54a-02dcd5b2182d

📥 Commits

Reviewing files that changed from the base of the PR and between bdb7382 and 085f257.

📒 Files selected for processing (3)

• scripts/build/CLAUDE.md
• scripts/build/codegen.ts
• test/internal/source-lints/build-codegen-source-lists.test.ts

---

_{Comment @coderabbitai help to get the list of available commands.}

[robobun]

Status: reproduced and fixed, waiting on CI.

• Reproduced on a warm linux-x64 debug tree: after adding and then deleting src/js/internal/zz_probe.ts and reconfiguring, ninja -d explain codegen/InternalModuleRegistry+enum.h reports no work to do and the header still contains InternalZzProbe. The bindgen, host-exports, bake and bun-error edges have the same shape; generate-classes, bindgenv2 and node-fallbacks do not (their file list is on the command line).
• With this branch the deletion re-runs bundle-modules in the same configure+build (explain names codegen/js-sources.txt), a no-change reconfigure is still no work to do, and cxx-sources.txt keeps its old mtime so existing trees do not re-run cppbind. Transcripts are in the description.
• Test: test/internal/source-lints/build-codegen-source-lists.test.ts (fails without the scripts/build/codegen.ts change, passes with it).

[robobun]

^{Updated 10:19 AM PT - Aug 13th, 2026}

✅ @robobun, your commit 085f257e807573a6030c720674fdd928e36dd2a2 passed in Build #94521! 🎉

---

🧪   To try this PR locally:

bunx bun-pr 38038

That installs a local version of the PR into your bun-38038 executable, so you can run:

bun-38038 --bun

[claude]

LGTM — generalizes the existing cxx-sources.txt pattern to the five other glob-driven codegen steps so ninja re-runs them when an input is deleted.

What was reviewed:

• sourceListFile() is a byte-for-byte extraction of the inline emitCppBind logic (same path cxx-sources.txt, same repo-relative/forward-slash/newline-terminated format), so existing trees don't spuriously re-run cppbind.
• The five new manifests are pure additions to implicitInputs; writeIfChanged keeps mtime stable on unchanged reconfigures, and the helper's own mkdirSync covers the first-configure case now that it runs before emitCppBind.
• The test follows the existing source-lints mockToolchain/resolveConfig pattern (windows-cross-config.test.ts), reads only the real packages/bun-error/package.json from the repo, and spawns nothing.

Extended reasoning...

Overview

This PR fixes a build-system incrementality gap: on a warm tree, deleting a file that a codegen step globs (e.g. rm src/js/internal/foo.ts) did not re-run that step, leaving stale generated code in the binary until some surviving input was edited. It touches only scripts/build/codegen.ts (build tooling, ~50 lines net), scripts/build/CLAUDE.md (one sentence), and adds a new hermetic test in test/internal/source-lints/.

The fix extracts the existing inline cxx-sources.txt write from emitCppBind into a shared sourceListFile() helper and applies it as an implicit input to five more steps: emitJsModules, emitBindgen, emitHostExports, emitBakeCodegen, emitBunError. Six previously-private emitters and the Ctx type are exported so the test can drive them without going through emitCodegen() (which would spawn bun for bindgenv2 list-outputs).

Security risks

None. This is configure-time build tooling that writes text manifests into the build directory and adds ninja edge inputs. It does not touch runtime code, network, auth, or anything that ships in the binary.

Level of scrutiny

Low-to-moderate. Build-system-only; a bug here could at worst cause spurious codegen re-runs (mitigated by writeIfChanged + restat=1) or, in the other direction, leave the incrementality gap unfixed. CI does clean builds, so it can't affect release artifacts. The mechanism is not new — it is the exact pattern emitCppBind has used for cxx-sources.txt all along, generalized to a helper. I verified the refactored emitCppBind produces the same filename and content format as before, so existing trees don't churn.

Other factors

• The PR description is unusually thorough: it includes real ninja -d explain transcripts demonstrating the bug and the fix, a synthetic ninja check confirming the command-line-carrying steps (generate-classes, bindgenv2, build-fallbacks) are unaffected, and tsc --noEmit verification.
• The test is hermetic (no ninja, no compiler, no subprocess), uses tempDir, and follows the established mockToolchain/resolveConfig pattern from windows-cross-config.test.ts in the same directory. It asserts both structure (each edge declares its manifest as an implicit input, manifest content matches the fed files) and behavior (mtime stable across an unchanged reconfigure; only the changed manifest is rewritten when a file is dropped). The author reports it fails with scripts/ stashed and with only the manifest wiring removed.
• The one repo-file dependency the test picks up transitively (packages/bun-error/package.json via emitBunInstall) exists and is a stable build input.
• No prior human or bot reviews to defer to; only a CodeRabbit rate-limit notice.

[claude]

LGTM — generalizes the existing cxx-sources.txt manifest pattern to the five other glob-driven codegen steps so ninja re-runs them when an input is deleted.

What was reviewed:

• sourceListFile() is a byte-for-byte extraction of the old emitCppBind inline logic (same cxx-sources.txt path/format, so warm trees don't re-run cppbind); mkdirSync now fires earlier via emitBunError but was never the first codegenDir writer anyway.
• Manifests are added as implicit inputs only — the codegen/esbuild rules don't read $in, so command lines are unchanged and no-op reconfigures stay no work to do (verified via writeIfChanged in fs.ts and the mtime assertion in the second test).
• The new test follows the existing windows-cross-config.test.ts pattern (mock toolchain + resolveConfig) and lives in source-lints/ per that dir's README; it pins both the edge wiring and the write-only-when-changed invariant.

Extended reasoning...

Overview

This PR fixes a build-system incrementality gap: deleting a file that a codegen step globs (e.g. rm src/js/internal/foo.ts) did not re-run that step on a warm tree, because ninja only re-runs an edge when an input is newer or the command line changed — a file that vanished from the input list is neither. The fix extracts the existing cxx-sources.txt manifest logic from emitCppBind into a sourceListFile() helper and applies it to the five other affected steps (emitJsModules, emitBindgen, emitHostExports, emitBakeCodegen, emitBunError). Each writes codegen/<name>-sources.txt at configure time via writeIfChanged and lists it as an implicit input, so a deletion becomes a manifest rewrite that ninja can see. Three files touched: scripts/build/codegen.ts (+62/-24), a new test in test/internal/source-lints/, and a one-line scripts/build/CLAUDE.md doc update.

Security risks

None. This is configure-time build-graph generation; it does not touch runtime code, user input, network, auth, or anything shipped in the binary. The only new I/O is writing five small text files under the build directory at configure time.

Level of scrutiny

Low-to-medium. The mechanism is not new — emitCppBind has done exactly this for cxx-sources.txt all along, and the helper preserves that path and content byte-for-byte (verified: same relative(cfg.cwd, p).replace(/\\\\/g, '/') + trailing newline), so existing trees keep their mtime and don't spuriously re-run cppbind. The five new manifests are pure additions to implicitInputs; the codegen/esbuild rules use $args rather than $in, so command lines are untouched and the only observable effect is the intended one. The PR description includes real ninja -d explain transcripts demonstrating both the bug and the fix, plus a synthetic check confirming the command-line-carrying steps (generate-classes, bindgenv2, node-fallbacks) already handle deletion via ninja's command-hash tracking and correctly do not need a manifest.

Other factors

• The test is well-placed (test/internal/source-lints/ per its README: build-script unit tests that never touch the bun binary) and follows the established windows-cross-config.test.ts pattern of mock Toolchain + resolveConfig. It emits the six steps into a temp dir, parses the ninja text back, and asserts (a) each edge declares its manifest as an implicit input, (b) every line in each manifest is also a tracked edge input (so edits still re-run as before), (c) an unchanged reconfigure leaves every manifest's content and mtime untouched, and (d) dropping one JS source rewrites only js-sources.txt. Hermetic, no subprocess, uses tempDir with using.
• Exporting Ctx and the six emitters is scoped to internal build scripts (not public API) and is justified in a comment: emitCodegen() as a whole spawns bun for bindgenv2 list-outputs, which the test needs to avoid.
• The one-time cost (five steps re-run once on trees configured before this change, when their manifests first appear) is disclosed in the description and is the expected behavior for any new implicit input.
• CI reports +0.0 KB on every platform binary, consistent with a change that only affects when codegen re-runs, not what it produces.