Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/runtime/module-resolution.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -309,7 +309,7 @@ Bun includes its own implementation of a few npm packages. When you import one o

A built-in module does not implement every API of the package it replaces. For example, the built-in `undici` exports `Agent`, `Pool`, `Client`, `ProxyAgent`, and `MockAgent` so that imports succeed. These classes do not manage connections, and `fetch` ignores the `dispatcher` option. `stream()`, `pipeline()`, `connect()`, and `upgrade()` throw an error. If your code needs an API that is missing, [open an issue](https://bun.com/issues).

Bun matches the whole specifier. Any other subpath, such as `undici/package.json`, resolves to the installed package. For a replaced specifier, `require.resolve()` and `import.meta.resolve()` return the name of the built-in module instead of a file path. For example, both `"undici"` and `"next/dist/compiled/undici"` resolve to `"undici"`.
Bun matches the whole specifier. Any other subpath, such as `undici/package.json`, resolves to the installed package. A replaced specifier with a query string, such as `ws?v=1`, does not resolve. `--preload` and the `preload` option of `Worker` load the built-in module too. For a replaced specifier, `require.resolve()` and `import.meta.resolve()` return the name of the built-in module instead of a file path. For example, both `"undici"` and `"next/dist/compiled/undici"` resolve to `"undici"`.

[`bun build`](/bundler) applies the same replacement when the target is `bun`, and leaves the import in the bundle. With `--target=node` or `--target=browser`, `bun build` bundles the copy in `node_modules`.

Expand Down
2 changes: 1 addition & 1 deletion src/js_parser_jsc/Macro.rs
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ impl MacroContext {

let input_specifier: &[u8] = 'brk: {
if let Some(replacement) = ModuleLoader::HardcodedModule::Alias::get(
import_record_path,
import_record_path_without_macro_prefix,
bun_ast::Target::Bun,
Default::default(),
) {
Expand Down
11 changes: 11 additions & 0 deletions src/jsc/ModuleLoader.rs
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,17 @@ pub fn exposed_internal_tag(spec: &[u8]) -> Option<(Vec<u8>, crate::ResolvedSour
Some((name, tag))
}

/// Whether `name` is a builtin module's: an alias-table key, or an `internal/` module under `--expose-internals`.
pub fn is_builtin(name: &[u8]) -> bool {
HardcodedModule::Alias::has(name, bun_ast::Target::Bun, Default::default())
|| exposed_internal_tag(name).is_some()
}

/// Whether a preload skips the resolver, which takes a builtin's name for a package. Any `node:` name skips it too.
pub fn preload_skips_resolver(name: &[u8]) -> bool {
name.starts_with(b"node:") || is_builtin(name)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) Users who preload a replaced npm package (--preload ws, -r abort-controller/polyfill, bunfig preload = ["undici"]) now get Bun's replacement module instead of the package they installed, silently. src/jsc/ModuleLoader.rs:213 makes preload_skips_resolver true for every alias-table key, including the 19 third-party replacement names, so src/runtime/jsc_hooks.rs:755 never calls resolve_and_auto_install and node_modules/ws is never read. Node -r ws runs the installed package; the base branch did too. Fix: limit the resolver skip to Node/Bun builtins (node_builtin keys, bun: keys, exposed internals) and keep replaced package names on the resolver path, at both src/runtime/jsc_hooks.rs:755 and src/jsc/web_worker.rs:331.

Why this was flagged

Trigger: bun --preload ws ./app.ts, bun -r abort-controller/polyfill ..., or bunfig preload = ["undici"] with the package installed in node_modules; this is the Node -r <pkg> migration pattern. On the base branch src/runtime/jsc_hooks.rs:753 only skipped the resolver for node: names, so the installed package resolved and ran. After the change HardcodedModule::Alias::has at src/jsc/ModuleLoader.rs:207 is true for ws, undici, node-fetch, abort-controller/polyfill, utf-8-validate, next/dist/compiled/* (src/resolve_builtins/HardcodedModule.rs:747-802), so preload_skips_resolver at src/jsc/ModuleLoader.rs:213 returns true and src/runtime/jsc_hooks.rs:756 passes the raw name to JSModuleLoader::import, which loads Bun's replacement. A patched or pinned ws/undici the user deliberately installed is bypassed with no message; Node and base Bun ran it. Remedy: skip the resolver for node_builtin / bun: / exposed-internal names only, leaving replaced package names on the resolver path.

Verification: src/jsc/ModuleLoader.rs:206-208 is_builtin is HardcodedModule::Alias::has(name, Target::Bun, ..), and BUN_ALIAS_MAP includes the "Thirdparty packages we override" entries at src/resolve_builtins/HardcodedModule.rs:747-802. src/jsc/ModuleLoader.rs:212-213 preload_skips_resolver returns true for those, so src/runtime/jsc_hooks.rs:755-756 hand the raw name to JSModuleLoader and node_modules/ws is never read.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed. The Notes of the PR body list this row as ruling 2 for a maintainer.

The reason for the current choice: a preload of ws now loads the module that import "ws" loads in the same program. The documentation says that Bun loads its own module for these specifiers "even when the package is installed in node_modules", so the program does not use the installed ws, patched or not. With no node_modules, main asks the registry for ws for the preload, runs it, and then the program imports the module of Bun.

The other choice keeps a replaced name with the resolver, for a preload and for a ?query. It needs a way to tell a replaced package from a builtin, and the alias table has no such field today. It is a small follow-up if a maintainer prefers it.

}

/// C++ entry point: whether `data[..len]` names a builtin module.
#[unsafe(no_mangle)]
unsafe extern "C" fn ModuleLoader__isBuiltin(data: *const u8, len: usize) -> bool {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 nit (optional): Maintainers now carry two copies of the "is this name a builtin" predicate that can drift apart. The new is_builtin at src/jsc/ModuleLoader.rs:206 and the existing ModuleLoader__isBuiltin at src/jsc/ModuleLoader.rs:221 compute the same answer, one through the BUN_ALIAS_MAP hash lookup and one through the linear bun_aliases_get scan flagged "PERF" at src/jsc/ModuleLoader.rs:171. Fix: have ModuleLoader__isBuiltin return is_builtin(str) so require.resolve.paths and Module._resolveLookupPaths share the single predicate; bun_aliases_get stays for its one remaining caller at src/runtime/jsc_hooks.rs:4677, which needs the Alias value.

Why this was flagged

The diff adds is_builtin at src/jsc/ModuleLoader.rs:206-209 as HardcodedModule::Alias::has(name, Target::Bun, Default::default()) || exposed_internal_tag(name).is_some(). The pre-existing C++ entry point ModuleLoader__isBuiltin at src/jsc/ModuleLoader.rs:218-222 evaluates bun_aliases_get(str).is_some() || exposed_internal_tag(str).is_some(). bun_aliases_get at src/jsc/ModuleLoader.rs:174-190 linearly scans BUN_ALIASES after the same stream_iter_alias_gated gate that Alias::get applies at src/resolve_builtins/HardcodedModule.rs:928, and BUN_ALIAS_MAP at src/resolve_builtins/HardcodedModule.rs:900 is built from the same BUN_ALIASES tables (line 830), so the two predicates are semantically identical today. No runtime behaviour differs; this is a duplicate-implementation concern under the repository's one-implementation rule, and the comment at src/jsc/ModuleLoader.rs:171-172 already asks for the hash-map form that is_builtin now provides. Nothing on the base branch prevents it, because the base had only the scan-based predicate.

Verification: nit. The PR adds a second "is this name a builtin" predicate beside an existing one that gives the identical answer. New is_builtin at src/jsc/ModuleLoader.rs:206-209 looks up BUN_ALIAS_MAP; the unchanged ModuleLoader__isBuiltin at ModuleLoader.rs:218-222 uses bun_aliases_get, which linearly scans the same BUN_ALIASES tables. Nothing breaks today, so this is cleanup-grade.

Expand Down
15 changes: 6 additions & 9 deletions src/jsc/VirtualMachine.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5067,7 +5067,7 @@ impl VirtualMachine {

/// Note: `is_a_file_path` is a runtime
/// arg to avoid duplicating the body for both monomorphizations.
pub(crate) fn _resolve(
fn _resolve(
&mut self,
ret: &mut ResolveFunctionResult,
specifier: &[u8],
Expand All @@ -5084,6 +5084,7 @@ impl VirtualMachine {
// note). Erase to `'static` to seat the result paths without threading
// a lifetime parameter through the VM.
let specifier: &'static [u8] = unsafe { bun_ptr::detach_lifetime(specifier) };
debug_assert!(!ModuleLoader::is_builtin(specifier));

// `Runtime.Runtime.Imports.{alt_name, Name}` are both `"bun:wrap"`
// (see js_parser/runtime.rs).
Expand All @@ -5103,14 +5104,6 @@ impl VirtualMachine {
ret.path = self.dupe_resolved_path(specifier);
return Ok(());
}
if let Some(result) = ModuleLoader::HardcodedModule::Alias::get(
specifier,
bun_ast::Target::Bun,
Default::default(),
) {
ret.path = result.path.as_bytes();
return Ok(());
}
if self.module_loader.eval_source.is_some()
&& (specifier.ends_with(bun_paths::path_literal!("/[eval]").as_bytes())
|| specifier.ends_with(bun_paths::path_literal!("/[stdin]").as_bytes()))
Expand All @@ -5134,6 +5127,10 @@ impl VirtualMachine {
let is_special_source = source == MAIN_FILE_NAME || Macro::is_macro_path(source);
let mut query_string: &[u8] = b"";
let normalized_specifier = normalize_specifier_for_resolution(specifier, &mut query_string);
// `path?v=2` names no module, as in Node: what the cut leaves is the builtin's name, not a package's.
if !query_string.is_empty() && ModuleLoader::is_builtin(normalized_specifier) {
return Err(crate::CrateError::ModuleNotFound);
}
let top_level_dir = self.top_level_dir();
let source_to_use: &[u8] = if !is_special_source {
if is_a_file_path {
Expand Down
5 changes: 2 additions & 3 deletions src/jsc/web_worker.rs
Original file line number Diff line number Diff line change
Expand Up @@ -327,9 +327,8 @@ impl WebWorker {
let mut preloads: Vec<Box<[u8]>> = Vec::with_capacity(preload_modules_len);
for module in preload_modules {
let utf8_slice = module.to_utf8();
// node: builtin specifiers skip the file resolver — the worker-side
// module loader resolves them.
if utf8_slice.slice().starts_with(b"node:") {
// The entry-point resolver takes a builtin's name for a package, or for a `./name` file.
if crate::module_loader::preload_skips_resolver(utf8_slice.slice()) {
preloads.push(utf8_slice.slice().to_vec().into_boxed_slice());
continue;
}
Expand Down
5 changes: 2 additions & 3 deletions src/runtime/jsc_hooks.rs
Original file line number Diff line number Diff line change
Expand Up @@ -751,9 +751,8 @@ unsafe fn load_preloads(vm: *mut VirtualMachine) -> bun_jsc::CrateResult<*mut JS
.strip_prefix(b"file://".as_slice())
.unwrap_or(preload_slice);

// node: builtin specifiers bypass the file resolver — JSModuleLoader
// resolves them internally, so `bun --import node:*` works like Node's.
let module_name = if normalized.starts_with(b"node:") {
// JSModuleLoader resolves a builtin's name: `bun --import path` works like Node's.
let module_name = if bun_jsc::module_loader::preload_skips_resolver(normalized) {
bun_core::String::from_bytes(normalized)
} else {
// ── resolve ─────────────────────────────────────────────────────
Expand Down
27 changes: 27 additions & 0 deletions test/bundler/transpiler/macro-test.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,33 @@ test("bun builtins can be used in macros", async () => {
expect(escapeHTML("abc!")).toBe("abc!");
});

// `macro:path` is the builtin, as `"path" with { type: "macro" }` is: not the package of that name.
test("a macro: import of a builtin's name does not run the package of that name in node_modules", async () => {
using dir = tempDir("macro-builtin-name", {
"node_modules/path/package.json": JSON.stringify({ name: "path", version: "1.0.0", main: "index.js" }),
"node_modules/path/index.js": `exports.basename = () => "node_modules/path";`,
"index.js": `
import { basename } from "macro:path";
import { basename as withAttribute } from "path" with { type: "macro" };
console.log(basename("/a/b.txt"), withAttribute("/a/b.txt"));
`,
});
await using proc = Bun.spawn({
cmd: [bunExe(), "index.js"],
env: bunEnv,
cwd: String(dir),
stdout: "pipe",
stderr: "pipe",
});
const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]);
// Debug builds print "[macro] call basename" to stdout first, so only the tail of stdout is matched.
expect({ stdout, stderr, exitCode }).toMatchObject({
stdout: expect.stringMatching(/(^|\n)b\.txt b\.txt\n$/),
stderr: "",
exitCode: 0,
});
});

test("latin1 string", () => {
expect(identity("©")).toBe("©");
});
Expand Down
148 changes: 147 additions & 1 deletion test/cli/run/preload-test.test.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { spawnSync } from "bun";
import { describe, expect, test } from "bun:test";
import { mkdirSync, realpathSync } from "fs";
import { bunEnv, bunExe } from "harness";
import { bunEnv, bunExe, isDebug, tempDir } from "harness";
import { tmpdir } from "os";
import { join } from "path";
const preloadModule = `
Expand Down Expand Up @@ -191,6 +191,152 @@ plugin({
expect({ stdout, stderr, exitCode }).toEqual({ stdout: "main ran\n", stderr: "", exitCode: 0 });
});

// A builtin's bare name is the builtin, as it is to `import` and to `node -r path`. The resolver would take it
// for a package. "abort-controller" is a package that Bun replaces with a built-in module.
describe("of a builtin by its bare name", () => {
const builtins = ["path", "fs/promises", "bun:sqlite", "abort-controller", "events"];
const packages = (...names) =>
Object.fromEntries(
names.flatMap(name => [
[`node_modules/${name}/package.json`, JSON.stringify({ name, version: "1.0.0", main: "index.js" })],
[`node_modules/${name}/index.js`, `console.log("node_modules/${name} ran");`],
]),
);

async function run(cwd, args) {
const requests = [];
await using registry = Bun.serve({
port: 0,
fetch(req) {
requests.push(new URL(req.url).pathname);
return new Response("{}", { status: 404, headers: { "content-type": "application/json" } });
},
});
await using proc = Bun.spawn({
cmd: [bunExe(), ...args, "main.js"],
env: {
...bunEnv,
NODE_NO_WARNINGS: undefined,
BUN_CONFIG_REGISTRY: registry.url.href,
NPM_CONFIG_REGISTRY: registry.url.href,
},
cwd,
stdout: "pipe",
stderr: "pipe",
});
const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]);
return { stdout, stderr, exitCode, requests };
}

test.each(["--preload", "--import", "--require", "bunfig.toml"])(
"%s does not run a file that the resolver has for that name",
async how => {
// A package that is no builtin, and a file, are still the resolver's.
const preloads = [...builtins, "lodashy", "./setup.js"];
using dir = tempDir("preload-builtin-name-resolver", {
...packages("path", "fs", "abort-controller", "lodashy"),
"node_modules/fs/promises.js": `console.log("node_modules/fs/promises.js ran");`,
"tsconfig.json": JSON.stringify({ compilerOptions: { baseUrl: "./src" } }),
"src/events.js": `console.log("src/events.js ran");`,
"setup.js": `console.log("setup.js ran");`,
"main.js": `console.log("main ran");`,
...(how === "bunfig.toml" ? { "bunfig.toml": `preload = ${JSON.stringify(preloads)}` } : {}),
});

const args = how === "bunfig.toml" ? [] : preloads.flatMap(name => [how, name]);
expect(await run(String(dir), args)).toEqual({
stdout: "node_modules/lodashy ran\nsetup.js ran\nmain ran\n",
stderr: "",
exitCode: 0,
requests: [],
});
},
);

test("--preload does not ask the registry for a package of that name", async () => {
using dir = tempDir("preload-builtin-name-autoinstall", { "main.js": `console.log("main ran");` });
const args = builtins.flatMap(name => ["--preload", name]);
expect(await run(String(dir), args)).toEqual({ stdout: "main ran\n", stderr: "", exitCode: 0, requests: [] });
});

// The name that goes to the resolver is the one without `file://`, so that name is the one to test.
test("--preload file://path does not run the package of that name", async () => {
using dir = tempDir("preload-builtin-name-file-url", {
...packages("path"),
"main.js": `console.log("main ran");`,
});
const { stdout, requests } = await run(String(dir), ["--preload", "file://path"]);
expect({ stdout, requests }).toEqual({ stdout: "main ran\n", requests: [] });
Comment on lines +268 to +269

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 nit (optional): The file://path preload test drops stderr and exitCode from its assertion, so a nonzero exit or a stray warning on that path goes unnoticed. test/cli/run/preload-test.test.js:268-269 destructures only { stdout, requests } while every sibling test in the block asserts the full { stdout, stderr, exitCode, requests } object. Fix: assert the combined object with stderr: "" and exitCode: 0, as the --preload does not ask the registry test at :259 does, or state in a comment why those two fields cannot be asserted here.

Why this was flagged

The test at test/cli/run/preload-test.test.js:263-270 spawns bun --preload file://path main.js through the run helper, which returns { stdout, stderr, exitCode, requests }. Line 268 keeps only stdout and requests, and line 269 compares only those. If load_preloads at src/runtime/jsc_hooks.rs:755 later regressed to print a warning to stderr or main.js exited nonzero after printing, the test still passes. The sibling tests at :248-252 and :259 assert stderr: "" and exitCode: 0; the base branch has none of these tests, and REVIEW.md asks for the combined { stdout, stderr, exitCode } object. No comment explains why the two fields are omitted.

Verification: nit. Trigger: any future regression on the --preload file://path path that prints to stderr or exits nonzero after "main ran" goes undetected by this test. The run helper at test/cli/run/preload-test.test.js:227-228 returns { stdout, stderr, exitCode, requests }, but the test at :268-269 keeps only const { stdout, requests } = await run(...).

});

const worker = preload => `
const worker = new Worker("./worker.js", { preload: ${JSON.stringify(preload)} });
worker.onmessage = e => {
console.log(e.data);
worker.terminate();
};
worker.onerror = e => {
console.log("error: " + e.message);
};
`;

// The `preload` option of a Worker is "equivalent to passing the `--preload` CLI argument". Its resolver also
// takes a bare name for a `./name` file: `util` is the builtin, and `./util` is the file.
test("Worker preload does not run a file that the resolver has for that name", async () => {
using dir = tempDir("preload-builtin-name-worker", {
...packages("path", "abort-controller", "lodashy"),
"util.ts": `console.log("util.ts ran");`,
"worker.js": `postMessage("main ran");`,
"main.js": worker(["path", "bun:sqlite", "abort-controller", "util", "./util", "lodashy"]),
});
expect(await run(String(dir), [])).toEqual({
stdout: "util.ts ran\nnode_modules/lodashy ran\nmain ran\n",
stderr: "",
exitCode: 0,
requests: [],
});
});

// A preload that is skipped passes the tests above too. Only the builtin `_tls_wrap` warns when it loads.
// It loads `node:tls`, which takes seconds in a debug build.
test.skipIf(isDebug).each(["--preload", "Worker preload"])("%s loads the builtin", async how => {
using dir = tempDir("preload-builtin-name-loads", {
...packages("_tls_wrap"),
"worker.js": `postMessage("main ran");`,
"main.js": how === "--preload" ? `console.log("main ran");` : worker(["_tls_wrap"]),
});
const { stdout, stderr, exitCode } = await run(String(dir), how === "--preload" ? [how, "_tls_wrap"] : []);
// "(node:<pid>) <warning>", then a hint that names the executable.
const warnings = stderr
.split("\n")
.map(line => line.replace(/^\(node:\d+\) /, ""))
.filter(line => line && !line.startsWith("(Use `"));
expect({ stdout, warnings, exitCode }).toEqual({
stdout: "main ran\n",
warnings: ["[DEP0192] DeprecationWarning: The _tls_wrap module is deprecated. Use `node:tls` instead."],
exitCode: 0,
});
});
});

// Under `--expose-internals` an `internal/` module is a builtin too, and `internal` is a package's name.
test("--preload of an internal/ module under --expose-internals", async () => {
using dir = tempDir("preload-builtin-name-expose-internals", {
"node_modules/internal/package.json": JSON.stringify({ name: "internal", version: "1.0.0" }),
"node_modules/internal/validators.js": `console.log("node_modules/internal/validators.js ran");`,
"main.js": `console.log("main ran");`,
});
await using proc = Bun.spawn({
cmd: [bunExe(), "--expose-internals", "--preload", "internal/validators", "main.js"],
env: bunEnv,
cwd: String(dir),
stdout: "pipe",
stderr: "pipe",
});
const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]);
expect({ stdout, stderr, exitCode }).toEqual({ stdout: "main ran\n", stderr: "", exitCode: 0 });
});

test("throws an error when preloaded module not found", async () => {
const bunfig = `preload = ["./bad-file.js"]`;

Expand Down
Loading
Loading