Skip to content

bundler: use require() for embedded sqlite modules in cjs output - #38173

Open
robobun wants to merge 4 commits into
mainfrom
farm/3b0a1eca/sqlite-embed-cjs-require
Open

robobun wants to merge 4 commits into
mainfrom
farm/3b0a1eca/sqlite-embed-cjs-require

Conversation

@robobun

@robobun robobun commented Aug 13, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • A program that embeds a sqlite database (import db from "./db.sqlite" with { type: "sqlite", embed: "true" }) fails at startup when built with --compile --format=cjs, or with --compile --bytecode (which defaults to cjs): SyntaxError: import.meta is only valid inside modules. The --bytecode build also prints error: Failed to generate bytecode for ./entry.js. Plain bun build --target=bun --format=cjs output fails the same way when run.
  • The module the bundler synthesizes for the database is import.meta.require(<embedded path>, { type: "sqlite" }).db (src/bundler/ParseTask.rs, sqlite branch of get_ast). It is built directly as an AST and handed to new_lazy_export_ast, which does not run the visit pass, so nothing rewrites the import.meta and the printer emits it verbatim.
  • cjs output for target bun is wrapped in // @bun @bun-cjs\n(function(exports, require, module, __filename, __dirname) { (src/bundler/linker_context/postProcessJSChunk.rs), and import.meta inside a function body is a SyntaxError.

Fix

  • When the output format is cjs, the synthesized call targets ERequireCallTarget instead of import.meta.require, so the module prints as require(<embedded path>, { type: "sqlite" }).db. Other formats keep import.meta.require.
  • This is correct because the printer prints ERequireCallTarget as a bare require for cjs output (require_ref is None there, src/bundler/LinkerContext.rs), which binds to the wrapper's require parameter: the module's own CommonJS require, the same function import.meta.require returns, and it accepts the { type } attributes object as its second argument. The napi loader a few lines below already builds its lazy export this way.
  • Other formats are left alone on purpose: esm and iife output for target bun is loaded as an ES module (// @bun), where import.meta.require works today, and for a lazy export ERequireCallTarget would print the runtime's __require without linking in its definition.
  • Verified with bun bd test test/bundler/bundler_bun.test.ts and bun bd test test/bundler/bundler_compile.test.ts -t EmbeddedSqlite:
    • bun/embedded-sqlite-file-cjs checks the printed output (@bun-cjs wrapper, var db_default = require(...), no import.meta) and runs it. bun/embedded-sqlite-file-cjs-dynamic-import does the same for the other shape the linker gives a lazy export (module.exports = require(...) inside a __commonJS wrapper), which a dynamic import produces.
    • compile/EmbeddedSqlite+cjs, +cjs+minify (syntax, whitespace and identifiers), and +bytecode run the compiled executable; the bytecode variant also checks that the bytecode cache is hit, which fails while bytecode generation fails.
    • bun/sqlite-file-cjs covers the non-embedded loader (type: "sqlite" without embed) in cjs output. That path keeps the import external, so it never emitted import.meta; the test locks that shape in and runs the bundle.
    • The five embedded tests fail on the released build (USE_SYSTEM_BUN=1) with the SyntaxError above; the pre-existing esm tests (bun/embedded-sqlite-file, compile/EmbeddedSqlite) pass before and after.
    • Manually confirmed with the debug build that esm, iife, and --compile --bytecode --format=esm output for the same entry still runs.

Background

  • Lazy export: for non-JS loaders (json, toml, sqlite, napi, ...) the bundler does not parse anything; it builds one expression and wraps it as the module's default export via new_lazy_export_ast. Such ASTs skip the parser's visit pass, so format-dependent rewrites that normally happen there have to be made when the expression is built.
  • ERequireCallTarget: the AST node for the require function itself. The printer emits it as the runtime's __require helper in esm/iife output and as the bare identifier require in cjs output.
  • @bun-cjs wrapper: cjs output for target bun is emitted as a function taking (exports, require, module, __filename, __dirname); Bun calls it with the module's CommonJS require, which is also what resolves embedded $bunfs paths in a compiled executable. --bytecode uses this format by default because bytecode is generated for a CommonJS program.

no test proof · iteration 0 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/bundler/bundler_compile.test.ts

The module synthesized for an embedded sqlite database called
import.meta.require(). With --format=cjs (which --bytecode defaults to)
the chunk is wrapped in the @bun-cjs function wrapper, where import.meta
is a SyntaxError, so compiled executables failed at startup and bytecode
generation failed. The synthesized AST never runs through the visit
pass, so nothing rewrote it. For cjs output, call the wrapper's require
parameter instead, the same way the napi loader already does.
@coderabbitai

coderabbitai Bot commented Aug 13, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 3 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 68ff964a-a212-4f96-849d-3a51e5bc1905

📥 Commits

Reviewing files that changed from the base of the PR and between b7a0431 and e17a37d.

📒 Files selected for processing (3)
  • src/bundler/ParseTask.rs
  • test/bundler/bundler_bun.test.ts
  • test/bundler/bundler_compile.test.ts

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Aug 13, 2026 •

Copy link
Copy Markdown
Collaborator Author

Reproduced on the released 1.4.0 build and on main (b7a0431032): bun build entry.js --compile --format=cjs (and --compile --bytecode) with an embedded sqlite import produces an executable that exits with SyntaxError: import.meta is only valid inside modules.; the non-compiled --target=bun --format=cjs output fails the same way.

Fix is in this PR. New tests: bun/embedded-sqlite-file-cjs and bun/embedded-sqlite-file-cjs-dynamic-import in test/bundler/bundler_bun.test.ts; compile/EmbeddedSqlite+cjs, +cjs+minify, +bytecode in test/bundler/bundler_compile.test.ts. All fail with USE_SYSTEM_BUN=1 and pass with bun bd test.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — small, well-reasoned fix that swaps import.meta.require for ERequireCallTarget in the sqlite lazy-export AST when output format is cjs, matching the napi loader's existing pattern. The one inline nit (non-embedded type: "sqlite" cjs coverage) is worth adding but non-blocking since both loaders share the same match arm.

Extended reasoning...

Overview

This PR fixes a SyntaxError when bundling embedded sqlite modules with --format=cjs (or --bytecode, which defaults to cjs). The bundler synthesizes a lazy-export AST for the sqlite loader that hard-codes import.meta.require(...), but cjs output for target bun is wrapped in the @bun-cjs function wrapper where import.meta is a SyntaxError. Since lazy-export ASTs skip the visit pass, nothing rewrites it. The fix branches on opts.output_format == Cjs at the AST construction site to emit ERequireCallTarget instead, which the printer renders as bare require (binding to the wrapper's parameter). Three files touched: ~20 lines in ParseTask.rs, plus ~70 lines of new tests in bundler_bun.test.ts and bundler_compile.test.ts.

Security risks

None. This is bundler AST construction for a synthesized module — no user input parsing, no allocation-size arithmetic, no auth/crypto/permissions.

Level of scrutiny

Low-to-medium. The change is narrow (one branch in one match arm), follows the neighboring napi loader's existing pattern (ParseTask.rs:1118 already uses ERequireCallTarget), and only affects a code path that was previously broken (threw SyntaxError at runtime). I verified the printer behavior at js_printer/lib.rs:3288-3299 — ERequireCallTarget prints as bare require when require_ref is None, which the PR description confirms is the case for cjs. I also checked why the fix is cjs-only: the __require linking check in generateCodeForLazyExport.rs:399 matches on the outermost ECall, but sqlite's root is E::Dot, so using ERequireCallTarget in esm/iife would print __require without linking its definition — the PR description states this explicitly and the code confirms it.

Other factors

Test coverage is solid: four new tests (one bun build cjs test with onAfterBundle output-shape assertions, three --compile variants including bytecode with a cache-hit check that fails while bytecode generation fails). The pre-existing esm test is preserved as the first entry in the new loop. The PR description confirms all four fail on USE_SYSTEM_BUN=1 and pass on the debug build. The one bug-hunter finding is a coverage nit (non-embedded Loader::Sqlite shares the arm at line 945 but has no cjs test) — valid, but the code path is literally identical after path_to_use is set, so there is no distinct logic to regress independently. Not blocking.

Comment thread test/bundler/bundler_bun.test.ts
@robobun

robobun commented Aug 13, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 3:36 PM PT - Aug 13th, 2026

✅ @robobun, your commit e17a37d35c4ca8e5a78f85c4ca99f9734e67d791 passed in Build #94834! 🎉


🧪   To try this PR locally:

bunx bun-pr 38173

That installs a local version of the PR into your bun-38173 executable, so you can run:

bun-38173 --bun

Comment thread src/bundler/ParseTask.rs Outdated
Comment thread src/bundler/ParseTask.rs Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — earlier feedback (the bun/sqlite-file-cjs sibling and the comment length) is addressed, and no new issues were found on this pass.

What was reviewed:

  • The cjs branch builds ERequireCallTarget exactly as the napi loader does at ParseTask.rs:1114; opts.output_format is the same field read elsewhere in the file.
  • Confirmed the pre-existing esm compile/EmbeddedSqlite case is preserved as the { suffix: "", options: {} } variant, and the bytecode variant asserts a disk-cache hit so it fails if bytecode generation regresses.
  • Checked the onAfterBundle regexes against the two linker shapes (hoisted var db_default = require(...) and module.exports = require(...) inside a __commonJS wrapper) — both anchor on the bare require identifier and rule out import.meta.
Extended reasoning...

Overview

The PR changes one branch in src/bundler/ParseTask.rs (Loader::SqliteEmbedded | Loader::Sqlite in get_ast): when the output format is cjs, the synthesized lazy-export module calls ERequireCallTarget instead of import.meta.require. The rest of the diff is tests: three new itBundled cases in bundler_bun.test.ts (embedded-cjs, embedded-cjs dynamic import, non-embedded-cjs) and a four-variant matrix replacing the single compile/EmbeddedSqlite test in bundler_compile.test.ts (esm, +cjs, +cjs+minify, +bytecode).

Security risks

None. This is codegen for a bundler-synthesized module; no untrusted input parsing, no auth/crypto, no memory management. The only behavioral change is which identifier the printer emits for the require call target.

Level of scrutiny

Medium — the bundler is a critical path, but this change is a ~15-line format-gated conditional that mirrors the napi loader immediately below it (ParseTask.rs:1112-1119), which already uses ERequireCallTarget unconditionally for the same lazy-export mechanism. The PR description explains why esm/iife keep import.meta.require (a lazy export's ERequireCallTarget would print __require without linking its definition), and that reasoning is consistent with how require_ref is set per format in LinkerContext.rs.

Other factors

Since my previous review, the author addressed both open threads: bun/sqlite-file-cjs was added (my nit), and the multi-line code comment was cut to one line (comment-cop, twice). All inline threads are resolved. Test coverage is thorough: both linker shapes for the lazy export (hoisted var and __commonJS-wrapped module.exports), both loaders (embedded and non-embedded), the compile matrix including identifier minification and bytecode with a disk-cache-hit assertion, and the pre-existing esm cases are preserved. The PR description records that the new tests fail under USE_SYSTEM_BUN=1 and that esm/iife/--bytecode --format=esm were manually re-checked. The bug hunter found nothing on this pass.

@robobun

robobun commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator Author

Heads-up from #39715: that PR makes cjs output for the bun target pass the real import.meta to the @bun-cjs wrapper, declared from what the printer emitted, so the sqlite module's import.meta.require(...) loads there as well. It no longer touches the sqlite branch of ParseTask.rs, so there are no shared lines with this PR. Its test (bun/ImportMetaFormatCjsEmbeddedSqlite) only runs the bundle, so the two can land in either order.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant