Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 29 additions & 4 deletions docs/bundler/macros.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -85,28 +85,53 @@ macro();

## Export condition "macro"

When shipping a library containing a macro to npm or another package registry, use the `"macro"` export condition to provide a version of your package exclusively for the macro environment.
When shipping a library containing a macro to npm or another package registry, use the `"macro"` export condition to provide a version of your package exclusively for macro imports.

```json title="package.json" icon="file-json"
{
"name": "my-package",
"exports": {
"macro": "./index.macro.js",
"import": "./index.js",
"require": "./index.js",
"default": "./index.js",
"macro": "./index.macro.js"
"default": "./index.js"
}
}
```

List `"macro"` first. The first condition that matches decides the file, and a macro import also matches `"import"` and `"default"`.

With this configuration, users can consume your package at runtime or at bundle-time using the same import specifier:

```ts title="index.ts" icon="/icons/typescript.svg"
import pkg from "my-package"; // runtime import
import { macro } from "my-package" with { type: "macro" }; // macro import
```

The first import resolves to `./node_modules/my-package/index.js`; Bun's bundler resolves the second to `./node_modules/my-package/index.macro.js`.
The first import resolves to `./node_modules/my-package/index.js`; Bun resolves the second to `./node_modules/my-package/index.macro.js`.

A macro runs in Bun, so a macro import resolves with the conditions `"macro"`, `"bun"`, and `"node"`, whatever the `target` of the build is. The `conditions` that you pass to the build also apply, and `"development"` applies when it applies to the other imports of the file. The `"macro"` condition applies to the macro import only. Imports inside the macro's own module resolve as they do in Bun's runtime, without `"macro"`.

<Note>
The `"macro"` condition requires Bun v1.4.5 or newer. Older versions resolve a macro import like any other import.
</Note>

To support older versions too, export the macro file under a subpath of its own and import it as `my-package/macro`:

```json title="package.json" icon="file-json"
{
"name": "my-package",
"exports": {
".": {
"macro": "./index.macro.js",
"import": "./index.js",
"require": "./index.js",
"default": "./index.js"
},
"./macro": "./index.macro.js"
}
}
```

## Execution

Expand Down
3 changes: 2 additions & 1 deletion docs/runtime/module-resolution.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,7 @@ Once it finds the `foo` package, Bun reads its `package.json` to determine the p
{
"name": "foo",
"exports": {
"macro": "./index.macro.js", // if the import is a macro import
"bun": "./index.js",
"node-addons": "./index.js", // unless --no-addons was passed
"node": "./index.js",
Expand All @@ -195,7 +196,7 @@ Once it finds the `foo` package, Bun reads its `package.json` to determine the p
}
```

Whichever of these conditions occurs _first_ in the `package.json` determines the package's entrypoint.
Whichever of these conditions occurs _first_ in the `package.json` determines the package's entrypoint. The `"macro"` condition only matches a macro import: see [macros](/bundler/macros#export-condition-macro).

Bun respects subpath [`"exports"`](https://nodejs.org/api/packages.html#subpath-exports) and [`"imports"`](https://nodejs.org/api/packages.html#imports).

Expand Down
2 changes: 1 addition & 1 deletion src/bundler/ThreadPool.rs
Original file line number Diff line number Diff line change
Expand Up @@ -550,7 +550,7 @@ impl Worker {
// behind `macro_context.data` (raw `*mut`, no `Drop` glue);
// `Transpiler` has no `Drop` impl, so `worker.data = None` below
// would strand it. Free both transpilers' boxes explicitly — the
// box only owns a `MacroMap` and a lazy `bun_alloc::Arena`, no JSC
// box only owns a `MacroMap`, a lazy `Resolver` and a lazy `bun_alloc::Arena`, no JSC
// handles, so the worker thread tearing it down is safe.
if let Some(data) = worker.data.as_mut() {
if let Some(ctx) = data.transpiler.macro_context.take() {
Expand Down
2 changes: 1 addition & 1 deletion src/bundler/entry_points.rs
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,7 @@ impl MacroEntryPoint {
entry.code_buffer[..label_len].copy_from_slice(macro_label_);

let code_len: usize = 'brk: {
if import_path.base == b"bun" {
if import_path.dir.is_empty() && import_path.filename == b"bun" {
let mut cursor = std::io::Cursor::new(&mut entry.code_buffer[label_len..]);
write!(
&mut cursor,
Expand Down
2 changes: 1 addition & 1 deletion src/bundler/options.rs
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ const DEFAULT_WILDCARD_PATTERNS: &[(&[u8], &[u8])] = &[
// (b"/blob:", b""),
];

fn default_wildcard_patterns() -> Vec<WildcardPattern> {
pub(crate) fn default_wildcard_patterns() -> Vec<WildcardPattern> {
DEFAULT_WILDCARD_PATTERNS
.iter()
.map(|(p, s)| WildcardPattern {
Expand Down
99 changes: 98 additions & 1 deletion src/bundler/transpiler.rs
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ impl<'a> Transpiler<'a> {
// The lazily-created `Box<bun_js_parser_jsc::Macro::MacroContext>` is
// process-lifetime by default, but
// worker VMs run `destroy()` on thread exit and would otherwise strand
// one box per worker. The box only owns a `MacroMap` and an optional
// one box per worker. The box only owns a `MacroMap`, a `Resolver` and an optional
// `bun_alloc::Arena` — no JSC handles — so freeing it from either
// worker section-5 teardown or main-thread `global_exit` is safe.
if let Some(ctx) = self.macro_context.take() {
Expand Down Expand Up @@ -1071,7 +1071,104 @@ fn resolver_bundle_options_subset(
}
}

/// The options of a resolver for the imports of macros, from the options of the resolver of the importing build.
#[cold]
#[inline(never)]
fn macro_resolver_options(
from: &resolver::options::BundleOptions,
transform: &api::TransformOptions,
) -> crate::Result<resolver::options::BundleOptions> {
use crate::options_impl::{Target, TargetExt};
use resolver::options as ropts;

// As in `from_api`: the defaults of the target, `node-addons`, then the user's `--conditions`.
let user_conditions: Vec<&[u8]> = transform.conditions.iter().map(|s| &**s).collect();
let mut conditions = options::ESMConditions::init(
Target::default_conditions_map()[Target::BunMacro],
transform.allow_addons.unwrap_or(true),
&user_conditions,
)?;
// "development" says how the build runs, not what it is for: a macro import has it when the importing resolver has it.
if from.conditions.import.contains(b"development") {
conditions.append_slice(&[b"development"])?;
}
// As in `from_api`: `--main-fields` replaces the list of the target.
let main_fields = if transform.main_fields.is_empty() {
ropts::owned_string_list(Target::default_main_fields_map()[Target::BunMacro])
} else {
transform.main_fields.iter().cloned().collect()
};

Ok(ropts::BundleOptions {
// A macro runs in Bun: what the target decides is as in Bun's runtime, with "macro", without the build's externals.
target: Target::BunMacro,
conditions: ropts::Conditions {
import: conditions.import,
require: conditions.require,
style: conditions.style,
},
main_fields,
main_fields_is_default: transform.main_fields.is_empty(),
external: ropts::ExternalModules {
patterns: options::default_wildcard_patterns(),
..Default::default()
},
packages: ropts::Packages::Bundle,
mark_builtins_as_external: false,
polyfill_node_globals: false,
framework: None,

// The rest is not about the target. It stays as the importing resolver has it.
jsx: from.jsx.clone(),
extra_cjs_extensions: from.extra_cjs_extensions.clone(),
global_cache: from.global_cache,
install: from.install,
load_package_json: from.load_package_json,
load_tsconfig_json: from.load_tsconfig_json,
main_field_extension_order: from.main_field_extension_order.clone(),
install_preference: from.install_preference,
preserve_symlinks: from.preserve_symlinks,
rewrite_jest_for_tests: from.rewrite_jest_for_tests,
tsconfig_override: from.tsconfig_override.clone(),
tsconfig_override_json: from.tsconfig_override_json.clone(),
production: from.production,
force_node_env: from.force_node_env,
output_dir: from.output_dir.clone(),
root_dir: from.root_dir.clone(),
public_path: from.public_path.clone(),
compile: from.compile,
supports_multiple_outputs: from.supports_multiple_outputs,
tree_shaking: from.tree_shaking,
allow_runtime: from.allow_runtime,
// A macro import is an `import` statement: it reads the `esm` extension lists, which no target changes.
extension_order: ropts::ExtensionOrder {
default: ropts::ExtensionOrderGroup {
default: from.extension_order.default.default.clone(),
esm: from.extension_order.default.esm.clone(),
},
node_modules: ropts::ExtensionOrderGroup {
default: from.extension_order.node_modules.default.clone(),
esm: from.extension_order.node_modules.esm.clone(),
},
css: from.extension_order.css.clone(),
},
})
}

impl<'a> Transpiler<'a> {
/// A resolver for the imports of macros, made from the resolver of the build that imports them.
pub fn macro_resolver(
importer: &Resolver<'a>,
transform: &api::TransformOptions,
) -> crate::Result<Box<Resolver<'a>>> {
let opts = macro_resolver_options(&importer.opts, transform)?;
// SAFETY: the new resolver has the lifetime of `importer`, so what `importer` borrows outlives it.
let mut resolver = Box::new(unsafe { Resolver::for_target(importer, importer.log, opts) });
// Bun's runtime takes "main" before "module".
resolver.prefer_module_field = false;
Ok(resolver)
}

/// Called by [`init_runtime_state`](../runtime/jsc_hooks.rs)
/// to write `vm.transpiler`. Builds on:
/// * [`options::BundleOptions::from_api`] — `bun_bundler::options`
Expand Down
49 changes: 37 additions & 12 deletions src/js_parser_jsc/Macro.rs
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ pub(crate) struct MacroContext {
pub(crate) macros: MacroMap,
pub(crate) remap: bun_ptr::BackRef<MacroRemap>,
pub(crate) javascript_object: JSValue,
/// Resolves the imports of macros: `resolver` made over for the environment a macro runs in. Lazy like `bump`.
pub(crate) macro_resolver: Option<Box<Resolver<'static>>>,
/// The AST takes lifetime-erased `&[u8]` arena slices (property keys /
/// UTF-16 string data / `from_blob` JSON sub-parse), so we own the backing arena here
/// — `MacroContext` is stored in the long-lived `Transpiler` and outlives
Expand Down Expand Up @@ -95,11 +97,41 @@ impl MacroContext {
transform_options: Arc::clone(&transpiler.options.transform_options),
remap: bun_ptr::BackRef::new(&transpiler.options.macro_remap),
javascript_object: JSValue::ZERO,
macro_resolver: None,
// Deferred until `call()` — see field doc.
bump: None,
}
}

/// Resolves the import of a macro for Bun, where the macro runs, and not for the output of the importing build.
fn resolve(
&mut self,
source_dir: &[u8],
specifier: &[u8],
) -> crate::Result<bun_resolver::Result> {
// SAFETY: `resolver` outlives `self` (see struct comment), and nothing resolves with it during this call.
let importer = unsafe { &*self.resolver };
let resolver = match &mut self.macro_resolver {
Some(resolver) => resolver,
slot @ None => slot.insert(Transpiler::macro_resolver(
importer,
&self.transform_options,
)?),
};
// A context that lasts longer than one module meets a new log for each module.
resolver.log = importer.log;
// `PackageJSON::parse` records the `npm:` aliases of a project through this. The importer gets it at its first auto-install.
resolver.package_manager = importer.package_manager;
// `require.extensions` can get an entry after the first macro call.
if resolver.opts.extra_cjs_extensions != importer.opts.extra_cjs_extensions {
resolver
.opts
.extra_cjs_extensions
.clone_from(&importer.opts.extra_cjs_extensions);
}
Ok(resolver.resolve(source_dir, specifier, bun_ast::ImportKind::Stmt)?)
}

pub(crate) fn call(
&mut self,
import_record_path: &[u8],
Expand Down Expand Up @@ -130,26 +162,19 @@ impl MacroContext {

debug_assert!(!is_macro_path(import_record_path_without_macro_prefix));

// SAFETY: `resolver` outlives `self` (see struct comment); uniquely
// accessed for the duration of this resolve call.
let resolver = unsafe { &mut *self.resolver };

let input_specifier: &[u8] = 'brk: {
if let Some(replacement) = ModuleLoader::HardcodedModule::Alias::get(
import_record_path,
import_record_path_without_macro_prefix,
bun_ast::Target::Bun,
Default::default(),
) {
break 'brk replacement.path.as_bytes();
}

let resolve_result = match resolver.resolve(
source_dir,
import_record_path_without_macro_prefix,
bun_ast::ImportKind::Stmt,
) {
let resolved = self.resolve(source_dir, import_record_path_without_macro_prefix);
let resolve_result = match resolved {
Ok(r) => r,
Err(bun_resolver::Error::ModuleNotFound) => {
Err(crate::Error::Resolver(bun_resolver::Error::ModuleNotFound)) => {
log.add_resolve_error(
Some(source),
import_range,
Expand All @@ -173,7 +198,7 @@ impl MacroContext {
bstr::BStr::new(import_record_path)
),
);
return Err(e.into());
return Err(e);
}
};
// The resolver's `Result` owns its path strings via the global `DirnameStore`
Expand Down
18 changes: 16 additions & 2 deletions src/jsc/RuntimeTranspilerStore.rs
Original file line number Diff line number Diff line change
Expand Up @@ -374,6 +374,10 @@
poll_ref: KeepAlive::default(),
resolved_source,
generation_number: self.generation_number.load(Ordering::SeqCst),
// SAFETY: this is the JS thread of `vm`, the only thread that writes the list.
extra_cjs_extensions: unsafe {
(*vm).transpiler.resolver.opts.extra_cjs_extensions.clone()

Check failure on line 379 in src/jsc/RuntimeTranspilerStore.rs

View workflow job for this annotation

GitHub Actions / cargo clippy

this public function might dereference a raw pointer but is not marked `unsafe`
},
parse_error: None,
work_task: WorkPoolTask {
node: Default::default(),
Expand Down Expand Up @@ -431,6 +435,8 @@
pub global_this: BackRef<JSGlobalObject>,
pub(crate) poll_ref: KeepAlive,
pub(crate) generation_number: u32,
/// `require.extensions` as of `transpile()`. The JS thread can replace the list of `vm.transpiler` while `run()` resolves.
pub(crate) extra_cjs_extensions: Box<[Box<[u8]>]>,
pub(crate) log: bun_ast::Log,
pub(crate) parse_error: Option<crate::CrateError>,
/// Moved out by `run_from_js_thread`; dropped with the slot otherwise.
Expand Down Expand Up @@ -700,7 +706,7 @@
// a bytewise copy. SAFETY: `vm.transpiler` is read via
// `addr_of!` (no `&VirtualMachine` formed); every internal raw pointer in the copy
// still targets memory owned by `vm.transpiler` (resolver caches, define, env) which
// outlives this stack frame; `vm.transpiler` is not concurrently mutated.
// outlives this stack frame. One thing the JS thread replaces meanwhile is the `require.extensions` list: see below.
Comment thread
robobun marked this conversation as resolved.
// The by-value copy must not be deinitialized; `ManuallyDrop` suppresses Drop so owned
// fields aren't double-freed against `vm.transpiler`.
let mut transpiler_storage =
Expand All @@ -715,6 +721,14 @@
unsafe { &mut *(&raw mut *transpiler_storage).cast::<Transpiler<'_>>() };
transpiler.set_arena(&arena);
transpiler.set_log(&raw mut log);
// The copy has the `require.extensions` list of `vm.transpiler`, which the JS thread frees when a script
// assigns an extension. SAFETY: the copy is never dropped, so the list of this job keeps one owner, the job.
unsafe {
ptr::write(
&raw mut transpiler.resolver.opts.extra_cjs_extensions,
ptr::read(&raw const self.extra_cjs_extensions),
)
};
// Note: the resolver already shares opts with the parent
// Transpiler via raw pointer; set_arena/set_log keep them in sync.
transpiler.macro_context = None;
Expand All @@ -729,7 +743,7 @@
// ~0.5 GB after 100k iterations). The owned `MimallocArena` inside is
// now lazy (`bump: Option<Arena>`, init on first `.call()`), so the
// per-iteration `mi_heap_new()` is gone; this guard just reclaims the
// small `Box`.
// small `Box` (and the macro resolver, when the module called a macro).
let _macro_ctx_guard =
scopeguard::guard(ptr::addr_of_mut!(transpiler.macro_context), |slot| {
// SAFETY: `slot` points into `transpiler_storage`, which is
Expand Down
13 changes: 13 additions & 0 deletions src/resolver/resolver.rs
Original file line number Diff line number Diff line change
Expand Up @@ -651,6 +651,19 @@ impl<'a> Resolver<'a> {
}
}

/// [`Self::for_worker`] with `opts` for another target. SAFETY: the same contract.
pub unsafe fn for_target(
from: &Resolver<'_>,
log: NonNull<bun_ast::Log>,
opts: options::BundleOptions,
) -> Resolver<'a> {
let care_about_browser_field = opts.target == options::Target::Browser;
// SAFETY: the caller upholds the contract of `for_worker`.
let mut resolver = unsafe { Self::for_worker(from, log, opts) };
resolver.care_about_browser_field = care_about_browser_field;
resolver
}

/// NOTE (Stacked Borrows): returns the RAW `*mut` (NOT `&'a mut`). A
/// `&'a mut` accessor would let two `fs()` calls manufacture coexisting
/// aliased unique refs to the same singleton (PORTING.md §Forbidden:
Expand Down
Loading
Loading