Skip to content

install: resolve a lockfile's workspace path when reading its package.json - #43405

Open
robobun wants to merge 1 commit into
mainfrom
robobun/7cdfdcc2/absolute-workspace-path-in-lockfile
Open

robobun wants to merge 1 commit into
mainfrom
robobun/7cdfdcc2/absolute-workspace-path-in-lockfile

Conversation

@robobun

@robobun robobun commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • An absolute workspace path in bun.lock or pnpm-lock.yaml aborts a debug build: panic: assertion failed: !is_input_absolute(input) (src/paths/Path.rs:911). A release build looks under <root>/<absolute path>, so --frozen-lockfile treats a workspace on disk as pruned.
  • A path longer than a path buffer aborts every build: panic: index out of bounds: the len is 4096 but the index is 4096. And bun add --filter leaves bun.lock unchanged for a workspace outside the root.
  • Cause: five sites build <root>/<workspace path>/package.json with AutoAbsPath::append, which takes only a relative input that fits and keeps .. as written.

Fix

Background

  • A workspace path is a member's directory relative to the root: a workspaces key in bun.lock, an importer key in pnpm-lock.yaml.
  • A pruned checkout (turbo prune) keeps the full bun.lock but only some members. --frozen-lockfile accepts a member that is missing on disk.
  • Path::append adds a relative component. Path::join resolves like path.resolve and normalizes ...
  • add --filter edits a cached package.json, keyed by its normalized path. The diff must ask for the same key.
Notes

Origin. Found while testing #43322. An absolute workspaces key stopped its debug build before its own check ran. No user has reported these symptoms.

History. Until #21059 the workspace diff built this path with joinAbsStringBuf(top_level_dir, [path, "package.json"]), which resolves. #21059 replaced it with AbsPath.append in a larger refactor, with no stated reason. #38333 copied the new shape into pruned_workspaces.rs. The pnpm migration has three more copies (pnpm.rs 744, 878, 2244), and two append(workspace_path) calls next to a join (1006, 1013).

Reproduction (no registry, no lifecycle script). $d/myapp is on disk and the root does not list it:

d=$(mktemp -d) && cd $d && mkdir -p myapp repo/tools/in
echo '{"name":"myapp"}' > myapp/package.json
cd repo
echo '{"name":"in","version":"1.0.0"}' > tools/in/package.json
echo '{"name":"root","workspaces":["tools/in"]}' > package.json
cat > bun.lock <<EOF
{
  "lockfileVersion": 1,
  "configVersion": 1,
  "workspaces": {
    "": { "name": "root" },
    "$d/myapp": { "name": "myapp" },
    "tools/in": { "name": "in", "version": "1.0.0" },
  },
  "packages": {
    "in": ["in@workspace:tools/in"],
    "myapp": ["myapp@workspace:$d/myapp"],
  }
}
EOF
bun install --frozen-lockfile
Input Before, debug Before, release 1.4.3-canary After
the bun.lock above, --frozen-lockfile assertion failed: !is_input_absolute(input) note: skipped 1 workspace listed in bun.lock but not on disk: "myapp", exit 0 error: lockfile had changes, but lockfile is frozen, exit 1, the same as for the key ../myapp
the same key, 100,000 bytes long, plain install index out of bounds index out of bounds (Windows: exit 3) the workspace is removed, Saved lockfile, exit 0
pnpm-lock.yaml importer '<root>/packages/a' the same assertion error: pnpm-lock.yaml lists importer '...' but '.../package.json' does not exist, Ignoring lockfile migrated lockfile from pnpm-lock.yaml, and the same bun.lock as for packages/a
pnpm-lock.yaml importer of 100,000 bytes index out of bounds index out of bounds the does not exist error, Ignoring lockfile, exit 0
root "workspaces": ["../x","../y"], bun add --filter x y@workspace:* x/package.json gets the dependency, bun.lock does not the same Saved lockfile, installed y@workspace:../y. remove --filter the same way

Why add --filter was affected. add --filter and remove --filter edit the package.json in workspace_package_json_cache. WorkspaceMap put it there under the normalized path <parent>/x/package.json. The diff asked for <root>/../x/package.json, missed, read the file from disk before the edit was written, and saw no change. For a path inside the root the two spellings are the same bytes, so nothing changes there.

Why the parser does not reject an absolute key. On Windows, relative() from C:\root to a directory on another drive returns the absolute path. With "workspaces": ["X:\\pkgx"] (checked with subst X:) the release build writes "X:/pkgx": {...} and "pkgx": ["pkgx@workspace:X:/pkgx"], and the next install reads it back with no changes. A parser that rejects the key would print InvalidLockfile and Ignoring lockfile on every install of such a project. #39357 and #39361 are about that setup. It is also why this PR adds no length rule to the parser: an Ignoring lockfile drops every pin, and the length of the key alone does not bound <root>/<key>/package.json.

The over-long key and #43322. #43322 refuses a workspace path that is too long with exit 1, in a pass that runs after the diff. So this PR only keeps the diff from aborting, and its test uses a plain install, where the diff removes the workspace before that pass can see it. The outcome under --frozen-lockfile is left to #43322.

Sites that still mishandle such a path. Not changed here. They run after the diff, in the linkers or in the --filter selection. #43322 puts its check in front of the linkers, and whether a workspace outside the root is legal at all needs a maintainer decision first.

  • isolated_install.rs:1771 and :1881 build AutoRelPath::from(workspace_path) to move an old <workspace>/node_modules aside. With this PR, a pruned workspace at an absolute or over-long key still aborts here when the isolated linker finds a node_modules without .bun.
  • isolated_install/Installer.rs:2661 and :2756 append the path of a workspace that is being installed. A bun.lock that gives a path-spec workspace a dependency on workspace:<absolute path> reaches :2756.
  • Over-long only: PackageInstaller.rs:1510 and PackageManagerDirectories.rs:1005 copy the path into a fixed buffer, and workspace_selection.rs:106 and :424 and add_remove_with_filter.rs:166, :186 and :196 use the unchecked join_abs_string_buf. install --frozen-lockfile --filter <name> with the 100,000-byte key still aborts in workspace_selection.rs. Report a path that does not fit a path buffer instead of aborting #43067 covers unchecked joins in general.
  • A Windows debug build stops a workspace on another drive even earlier, at debug_assert!(!bun_paths::is_absolute(path.slice(buf))) (Package.rs:2017), when it reads package.json.

Self-review. A review of the first draft found that the same two panics still fired from the pnpm copies of the idiom, that the Package.rs change altered add --filter and remove --filter with no test, that a test pinned an over-long outcome which #43322 decides differently, and that two claims in the draft were false (that the linker sites only see paths from package.json on POSIX, and that the problem there is Windows-only). All four are fixed in this version. It also proposed to land this as the base of #43322. The two diffs do not overlap in src/, so either order works.

Tests. Release build before (USE_SYSTEM_BUN=1): 5 of the 7 fail. The two that pass (absolute key and missing from disk, absolute key and plain install) fail only on a debug build, where the child aborts. Debug build before: all 7 fail. After: all 7 pass on Linux debug/ASAN and on Windows x64 debug. On Windows the add --filter test uses the hoisted linker, because the isolated linker there fails to link the dependencies of a workspace outside the root (ENOENT, also on 1.4.3-canary with a plain install).

Suites run on the Linux debug build with this change: frozen-lockfile-pruned.test.ts (105 pass), bun-add-filter.test.ts (124), migration/pnpm-lock-migration.test.ts (7), migration/pnpm-lock-v9.test.ts (84), migration/pnpm-comprehensive.test.ts (5), migration/pnpm-migration.test.ts (3), migration/pnpm-migration-complete.test.ts (1), migration/migrate.test.ts (129), bun-workspaces.test.ts (82), bun-lock.test.ts (40), bad-workspace.test.ts (14), catalogs.test.ts (89), bun-prune.test.ts (121), bun-workspaces-self-contained.test.ts (24), bun-remove.test.ts (14), isolated-install.test.ts (85). On Windows x64 debug: the three changed files (105, 124, 7).


no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/cli/install/frozen-lockfile-pruned.test.ts, test/cli/install/bun-add-filter.test.ts

….json

The pruned-workspace check, the workspace diff and the pnpm migration
built `<root>/<workspace path>/package.json` with `Path::append`.
`append` accepts only a relative input that fits the buffer. A
`workspaces` key in bun.lock and an importer key in pnpm-lock.yaml are
unchecked: they can be absolute, and they can have any length.

All of these sites now call `lockfile::workspace_package_json_path`. It
joins the path with the length-checked `AutoAbsPathChecked::join`, the
resolution that the workspace diff used before #21059.

The join also normalizes `..`, so the diff reads a workspace outside
the root through the same cache entry that `add --filter` and
`remove --filter` edit, and bun.lock now follows those edits.
@robobun

robobun commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status

  • Reproduced on main (367d939), Linux x64 and Windows x64. The input is a bun.lock with an absolute workspaces key that the root package.json does not list, then bun install --frozen-lockfile. The debug build aborts with panic: assertion failed: !is_input_absolute(input). The 1.4.3-canary release build prints note: skipped 1 workspace listed in bun.lock but not on disk for a workspace that is on disk. The exact steps are in the description, under Notes.
  • The fix and 7 tests are in this PR (install: resolve a lockfile's workspace path when reading its package.json #43405).
  • CI (build 118198): 180 of 181 jobs pass, and the 7 new tests pass on every lane (Linux, the ASAN lane, Windows x64 and aarch64, macOS x64 and aarch64). The one red job is test/bake/deinitialization.test.ts on alpine aarch64. This PR does not touch that code, and the same test fails on 6 of the last 8 builds of main. The diff is ready for review.

@coderabbitai

coderabbitai Bot commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: oven-sh/bun/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 8f8c57f6-069f-41b6-8bd7-85bef9a5efbb

📥 Commits

Reviewing files that changed from the base of the PR and between 26e7a4b and d2c4e8a.

📒 Files selected for processing (7)
  • src/install/lockfile.rs
  • src/install/lockfile/Package.rs
  • src/install/lockfile/pruned_workspaces.rs
  • src/install/pnpm.rs
  • test/cli/install/bun-add-filter.test.ts
  • test/cli/install/frozen-lockfile-pruned.test.ts
  • test/cli/install/migration/pnpm-lock-migration.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 0 remain after this review.


Walkthrough

Changes

Workspace path validation

Layer / File(s) Summary
Shared workspace manifest path resolution
src/install/lockfile.rs, src/install/lockfile/Package.rs, src/install/lockfile/pruned_workspaces.rs
Workspace package.json paths now use a checked path helper. Failed path resolution skips update mapping or marks the workspace as missing.
pnpm migration path integration
src/install/pnpm.rs
pnpm manifest lookups and workspace link comparisons now use validated joined paths. Invalid paths produce migration errors.
Relocated and oversized workspace regression coverage
test/cli/install/bun-add-filter.test.ts, test/cli/install/frozen-lockfile-pruned.test.ts, test/cli/install/migration/pnpm-lock-migration.test.ts
Tests cover external, absolute, relocated, and oversized workspace paths across install, filtered changes, pruning, and pnpm migration.

Suggested reviewers: jarred-sumner

Priority: ➖ Normal

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the main change: resolving workspace paths when reading package.json from the lockfile.
Description check ✅ Passed The description explains the problem, root cause, fix, affected behavior, regression coverage, and verification results. It does not use the exact template headings, but it provides the required infor…

Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline finding, I also checked the unchecked abs_path.join(&[path]) at src/install/pnpm.rs:881 — every non-root importer key passes the checked workspace_package_json_path join (same path plus /package.json) in the importer loop above before this runs, so the shorter join cannot overflow. The workspace_is_missing_on_disk change returning "missing" on an over-long key is consistent with the PR's stated intent (a path that does not fit counts as missing) and only affects pruning on a plain install.

Extended reasoning...

The one posted finding is a pre-existing, same-class overflow on the pnpm link: value, which the inline comment covers. Separately I verified the comment on the unchecked join at pnpm.rs:881: path is an importer key that equals a workspace_paths entry, and every such key was already built with workspace_package_json_path (a strictly longer path) in the importer loop, with an error return on overflow, so the claim in the comment holds. The Diff early break 'update_mapping false and workspace_is_missing_on_disk returning true on None both degrade to "workspace removed/changed" rather than a panic, matching the described behavior. The new tests exercise absolute keys, an over-long key, and add --filter outside the root; I did not independently run them.

Comment thread src/install/pnpm.rs
@robobun

robobun commented Sep 19, 2026

Copy link
Copy Markdown
Collaborator Author

This PR also fixes a reported abort that comes in through bun.lock, not through pnpm.

bun install aborts when a workspaces key in bun.lock is 4,080 bytes or longer:

rm -rf wk && mkdir -p wk/packages/a && cd wk
echo '{"name":"root","workspaces":["packages/*"]}' > package.json
echo '{"name":"a","version":"1.0.0"}' > packages/a/package.json
K=$(printf 'd/%.0s' $(seq 2048))d
printf '{\n  "lockfileVersion": 1,\n  "workspaces": {\n    "": { "name": "root" },\n    "%s": { "name": "w", "version": "1.0.0" }\n  },\n  "packages": {\n    "w": ["w@workspace:%s"]\n  }\n}\n' "$K" "$K" > bun.lock
bun install

On 1.4.0, 1.4.2 and main (9b7c982): panic: index out of bounds: the len is 4096 but the index is 4096, exit 134. On 1.3.14: 1 package installed, exit 0. The site is workspace_is_missing_on_disk (src/install/lockfile/pruned_workspaces.rs:21), which this PR routes through workspace_package_json_path.

I verified this branch (d2c4e8a) with a debug build:

  • plain install: 1 package installed, exit 0
  • --frozen-lockfile on a lockfile that is otherwise in sync: note: skipped 1 workspace listed in bun.lock but not on disk: "gone", exit 0, the lockfile is unchanged

I wrote five test cases for this door before I found this PR. They are on robobun/b554377f/lockfile-workspace-path-buffer-tests as one commit on top of this branch (89bab66), and all 19 tests in test/cli/install/bad-workspace.test.ts pass there. They add, next to the existing workspaces entries block in that file:

  • a plain install with a 100,000 byte path in bun.lock, which drops the workspace
  • --frozen-lockfile, which skips the workspace and keeps the lockfile byte for byte
  • the three paths one byte below, exactly, and one byte above the path buffer size

Four of the five fail on bun 1.4.3 (the child aborts). Cherry-pick the commit if you want them here. I am not opening a separate PR.

@robobun

robobun commented Sep 21, 2026

Copy link
Copy Markdown
Collaborator Author

A note for the rebase, in case #43537 lands first.

The two branches are each clean onto main (8cc0397). Together they conflict in one hunk of src/install/pnpm.rs, at the workspace: link site (line 1005 on main). Keep the join_top_level_dir call of #43537 in that hunk.

The side of this PR keeps the unchecked AutoAbsPath::join there. That join aborts on 1.4.3-canary (367d939) for specifier: workspace:* with version: link:<4,200 bytes>: panic: range end index 4230 out of range for slice of length 4095, exit 134. The checked join of #43537 also resolves an absolute path, so nothing of this PR is lost in that hunk.

The combined merge with that resolution is already built and tested: #43537 (comment). The test files of both PRs pass on it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant