Skip to content
Closed
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions docs/pm/workspaces.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,36 @@ Workspaces have a few major benefits.
- **Bun can de-duplicate dependencies.** If `a` and `b` share a common dependency, Bun _hoists_ it to the root `node_modules` directory. This saves disk space and minimizes the "dependency hell" of multiple versions of a package installed at once.
- **Run scripts in multiple packages.** Use the [`--filter` flag](/pm/filter) to run `package.json` scripts in several packages at once, or `--workspaces` to run scripts across all workspaces.

## Self-contained workspaces

With the hoisted linker, dependencies shared by several workspaces are hoisted to the root
`node_modules`. Some tools cannot follow that: Electron packagers and serverless bundlers walk,
prune and repackage *one workspace's* `node_modules` and expect every dependency to be physically
present under it. Mark such a workspace as self-contained, either in its own `package.json`
(the same key Yarn uses; only the `"workspaces"` value is recognized):

```json title="apps/desktop/package.json" icon="file-json"
{
"name": "desktop",
"installConfig": { "hoistingLimits": "workspaces" }
}
```

or from the root:

```toml title="bunfig.toml" icon="settings"
[install]
selfContainedWorkspaces = ["apps/desktop"] # workspace paths or names
```

For that workspace `bun install` then behaves as a hoisting barrier — nothing it depends on,
directly or transitively (including through other workspaces it depends on), is placed above
`apps/desktop/node_modules`, so that directory is a complete tree — and materializes those
packages as real copies instead of hardlinks / clones from the cache, so tools that rewrite them
cannot affect the cache or other projects. All other workspaces keep hoisting to the root as usual.
This setting has no effect with the isolated linker, where every package already resolves only its
own dependencies.

## Share versions with Catalogs

When many packages need the same dependency versions, define those versions once
Expand Down
9 changes: 9 additions & 0 deletions docs/runtime/bunfig.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -514,6 +514,15 @@ Valid values are:
| `"offline"` | Skip staleness checks and resolve packages from the local cache. Equivalent to `--prefer-offline`. |
| `"latest"` | Always check npm for the latest matching versions. Equivalent to `--prefer-latest`. |

### `install.selfContainedWorkspaces`

Workspaces (by path or name) that must have a complete, physical `node_modules` of their own when using the hoisted linker — see [Self-contained workspaces](/pm/workspaces#self-contained-workspaces). A workspace can also opt in itself with `"installConfig": { "hoistingLimits": "workspaces" }` in its `package.json`. Default `[]`.

```toml title="bunfig.toml" icon="settings"
[install]
selfContainedWorkspaces = ["apps/desktop"]
```

### `install.frozenLockfile`

When `true`, `bun install` does not update `bun.lock`. Default `false`. If `package.json` and the existing `bun.lock` disagree, the install errors.
Expand Down
26 changes: 26 additions & 0 deletions src/bunfig/bunfig.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1565,6 +1565,32 @@ impl<'a> Parser<'a> {
install.hoist = Some(v);
}

if let Some(list) = install_obj.get(b"selfContainedWorkspaces") {
match &list.data {
ExprData::EArray(arr) => {
let raw = arr.items.slice();
let mut out: Vec<Box<[u8]>> = Vec::with_capacity(raw.len());
for p in raw {
self.expect_string(p)?;
out.push(estring_to_owned(
p.data
.e_string()
.expect("infallible: variant checked")
.get(),
self.bump,
));
}
install.self_contained_workspaces = Some(out);
}
_ => {
self.add_error(
list.loc,
b"Expected an array of workspace paths or names for selfContainedWorkspaces",
)?;
}
}
}

Ok(())
}

Expand Down
13 changes: 12 additions & 1 deletion src/install/PackageInstaller.rs
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,10 @@ pub struct PackageInstaller<'a> {
pub(crate) successfully_installed: Bitset,
pub(crate) command_ctx: Command::Context<'a>,
pub(crate) current_tree_id: lockfile::tree::Id,
/// Trees that live under a self-contained workspace: packages there are copied
/// (real files) rather than hardlinked/cloned/symlinked from the cache, so tools
/// that walk, prune or rewrite that node_modules cannot reach the shared cache.
pub(crate) copy_trees: Bitset,

// fields used for running lifecycle scripts when it's safe
//
Expand Down Expand Up @@ -1869,10 +1873,17 @@ impl<'a> PackageInstaller<'a> {
break 'result result;
}

let method = if (self.current_tree_id as usize) < self.copy_trees.bit_length()
&& self.copy_trees.is_set(self.current_tree_id as usize)
{
package_install::Method::Copyfile
} else {
installer.get_install_method()
};
break 'result installer.install(
self.skip_delete,
&destination_dir,
installer.get_install_method(),
method,
resolution.tag,
);
}
Expand Down
16 changes: 16 additions & 0 deletions src/install/PackageManager.rs
Original file line number Diff line number Diff line change
Expand Up @@ -730,6 +730,20 @@ impl PackageManager {
unsafe { &*get() }
}

/// `get()` for code that may also run before the manager exists (helpers that
/// only consult options when an install is in progress).
#[inline]
pub fn try_get() -> Option<&'static PackageManager> {
let p = get();
if p.is_null() {
None
} else {
// SAFETY: same as `get()` — non-null means `allocate_package_manager()`
// ran and the singleton lives for the process.
Some(unsafe { &*p })
}
}

/// Associated-fn spelling that forwards to the free [`init`] so callers
/// can write `PackageManager::init(ctx, cli, subcommand)`.
#[inline]
Expand Down Expand Up @@ -1408,6 +1422,7 @@ fn overlay_bunfig_install(install: &mut Api::BunInstall, bunfig: Api::BunInstall
public_hoist_pattern,
hoist_pattern,
hoist,
self_contained_workspaces,
} = bunfig;

if let Some(registry) = default_registry {
Expand Down Expand Up @@ -1462,6 +1477,7 @@ fn overlay_bunfig_install(install: &mut Api::BunInstall, bunfig: Api::BunInstall
public_hoist_pattern,
hoist_pattern,
hoist,
self_contained_workspaces,
);
}

Expand Down
12 changes: 12 additions & 0 deletions src/install/PackageManager/PackageManagerOptions.rs
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,11 @@ pub struct Options {
/// fallback (pnpm's `hoist=false`); takes precedence over `hoist_pattern`.
pub(crate) hoist: bool,

/// Workspaces (path or name) whose node_modules must be complete and physical:
/// a hoisting barrier + copy materialization. Also enabled per workspace by
/// `"installConfig": { "hoistingLimits": "workspaces" }` in its package.json.
pub self_contained_workspaces: &'static [&'static [u8]],

// Security scanner module path
pub security_scanner: Option<&'static [u8]>,

Expand Down Expand Up @@ -156,6 +161,7 @@ impl Default for Options {
public_hoist_pattern: None,
hoist_pattern: None,
hoist: true,
self_contained_workspaces: &[],
security_scanner: None,
minimum_release_age_ms: None,
minimum_release_age_excludes: None,
Expand Down Expand Up @@ -475,6 +481,12 @@ impl Options {
self.hoist = hoist;
}

if let Some(list) = &config.self_contained_workspaces {
let leaked: Vec<&'static [u8]> = list.iter().map(|e| leak_static(e)).collect();
self.self_contained_workspaces =
&*bun_core::heap::release(leaked.into_boxed_slice());
}

if let Some(security_scanner) = config.security_scanner.as_deref() {
self.security_scanner = Some(leak_static(security_scanner));
self.do_.set(Do::PREFETCH_RESOLVED_TARBALLS, false);
Expand Down
7 changes: 7 additions & 0 deletions src/install/PackageManager/install_with_manager.rs
Original file line number Diff line number Diff line change
Expand Up @@ -417,6 +417,13 @@ pub fn install_with_manager(
lf.workspace_versions.insert(*key, version);
}
}
// …and which workspaces asked for a self-contained node_modules
{
lf.self_contained_workspaces.clear_retaining_capacity();
for key in lockfile.self_contained_workspaces.keys() {
lf.self_contained_workspaces.put(*key, ())?;
}
}

// Update patched dependencies
{
Expand Down
13 changes: 13 additions & 0 deletions src/install/hoisted_install.rs
Original file line number Diff line number Diff line change
Expand Up @@ -407,6 +407,19 @@ pub(crate) fn install_hoisted_packages(
folder_path_buf: bun_paths::PathBuffer::uninit(),
current_tree_id: tree::INVALID_ID,
pending_lifecycle_scripts: Vec::new(),
copy_trees: {
let self_contained = this.lockfile.self_contained_workspace_ids();
let mut set = Bitset::init_empty(trees_count)?;
if !self_contained.is_empty() {
for tid in 0..trees_count {
let owner = this.lockfile.owning_workspace_of_tree(tid as tree::Id);
if owner != 0 && self_contained.contains(&owner) {
set.set(tid);
}
}
}
set
},
};
};

Expand Down
66 changes: 66 additions & 0 deletions src/install/lockfile.rs
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,11 @@ pub struct Lockfile {
pub(crate) scripts: Scripts,
pub(crate) workspace_paths: NameHashMap,
pub workspace_versions: VersionHashMap,
/// Workspaces (by name hash) whose package.json declares
/// `installConfig.hoistingLimits = "workspaces"`; unioned with
/// `install.selfContainedWorkspaces` at hoist time. Not serialized: it is
/// re-derived from the manifests on every install.
pub self_contained_workspaces: ArrayHashMap<PackageNameHash, (), ArrayIdentityContextU64>,

/// Optional because `trustedDependencies` in package.json might be an
/// empty list or it might not exist
Expand Down Expand Up @@ -684,6 +689,7 @@ impl Lockfile {
self.trusted_dependencies = None;
self.workspace_paths = NameHashMap::default();
self.workspace_versions = VersionHashMap::default();
self.self_contained_workspaces = ArrayHashMap::default();
self.overrides = OverrideMap::default();
self.catalogs = CatalogMap::default();
self.patched_dependencies = PatchedDependenciesMap::default();
Expand Down Expand Up @@ -814,6 +820,57 @@ impl Lockfile {

/// Does this tree id belong to a workspace (including workspace root)?
/// TODO(dylan-conway) fix!
/// Workspace packages whose node_modules must be self-contained: listed in
/// `install.selfContainedWorkspaces` (by path or name) or declaring
/// `"installConfig": { "hoistingLimits": "workspaces" }` in their manifest
/// (recorded in `self_contained_workspaces` while the workspaces were parsed).
pub(crate) fn self_contained_workspace_ids(&self) -> Vec<PackageID> {
let configured: &[&[u8]] = crate::PackageManager::try_get()
.map(|pm| pm.options.self_contained_workspaces)
.unwrap_or(&[]);
if configured.is_empty() && self.self_contained_workspaces.count() == 0 {
return Vec::new();
}
let mut out = Vec::new();
let buf = self.buffers.string_bytes.as_slice();
let pkgs = self.packages.slice();
for (i, res) in pkgs.items_resolution().iter().enumerate() {
if res.tag != crate::resolution::Tag::Workspace {
continue;
}
let path = res.workspace().slice(buf);
let name = pkgs.items_name()[i].slice(buf);
let name_hash = pkgs.items_name_hash()[i];
if self.self_contained_workspaces.contains(&name_hash)
|| configured.iter().any(|c| {
let c = bun_core::strings::without_trailing_slash(c);
c == path || c == name
})
{
out.push(i as PackageID);
}
}
out
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved

/// The workspace (or root, 0) package that owns tree `id`: walk up until a
/// workspace/root tree is reached and return the package it stands for.
pub(crate) fn owning_workspace_of_tree(&self, mut id: tree::Id) -> PackageID {
while id != 0 && (id as usize) < self.buffers.trees.len() {
let t = &self.buffers.trees[id as usize];
let dep_id = t.dependency_id;
if (dep_id as usize) < self.buffers.dependencies.len()
&& self.buffers.dependencies[dep_id as usize]
.behavior
.is_workspace()
{
return self.buffers.resolutions[dep_id as usize];
}
id = t.parent;
}
0
}

pub(crate) fn is_workspace_tree_id(&self, id: tree::Id) -> bool {
id == 0
|| self.buffers.dependencies[self.buffers.trees[id as usize].dependency_id as usize]
Expand Down Expand Up @@ -1090,6 +1147,9 @@ impl Lockfile {
new.workspace_versions.re_index()?;
new.workspace_paths.re_index()?;
}
for key in old.self_contained_workspaces.keys() {
new.self_contained_workspaces.put(*key, ())?;
}
}

// When you run `"bun add react"
Expand Down Expand Up @@ -1322,8 +1382,10 @@ impl Lockfile {
// `ParentRef::new` captures `SharedReadOnly` provenance from `&*self`,
// which is exactly what `Builder` needs (it only ever `Deref`s); the
// `Builder` does not outlive this `&mut self` borrow.
let self_contained = self.self_contained_workspace_ids();
let lockfile_ref = bun_ptr::ParentRef::<Lockfile>::new(&*self);
let mut builder = tree::Builder::<METHOD> {
self_contained,
queue: tree::TreeFiller::init(),
resolution_lists: slice.items_resolutions(),
resolutions: self.buffers.resolutions.as_mut_slice(),
Expand Down Expand Up @@ -1986,6 +2048,7 @@ impl Lockfile {
trusted_dependencies: None,
workspace_paths: NameHashMap::default(),
workspace_versions: VersionHashMap::default(),
self_contained_workspaces: ArrayHashMap::default(),
overrides: OverrideMap::default(),
catalogs: CatalogMap::default(),
meta_hash: ZERO_HASH,
Expand Down Expand Up @@ -2342,6 +2405,7 @@ impl Lockfile {
workspace_versions: &mut self.workspace_versions,
patched_dependencies: &mut self.patched_dependencies,
trusted_dependencies: &mut self.trusted_dependencies,
self_contained_workspaces: &mut self.self_contained_workspaces,
},
)
}
Expand Down Expand Up @@ -2395,6 +2459,8 @@ pub(crate) struct LockfileFields<'a> {
pub(crate) workspace_versions: &'a mut VersionHashMap,
pub(crate) patched_dependencies: &'a mut PatchedDependenciesMap,
pub(crate) trusted_dependencies: &'a mut Option<TrustedDependenciesSet>,
pub(crate) self_contained_workspaces:
&'a mut ArrayHashMap<PackageNameHash, (), ArrayIdentityContextU64>,
}

// ────────────────────────────────────────────────────────────────────────────
Expand Down
5 changes: 5 additions & 0 deletions src/install/lockfile/Package.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2911,6 +2911,11 @@ impl Package<u64> {
.workspace_versions
.put(external_name.hash, version)?;
}
if entry.hoisting_limits {
lockfile
.self_contained_workspaces
.put(external_name.hash, ())?;
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
}
}
} else {
Expand Down
Loading
Loading