Skip to content

install: resolve peers provided by file: dependencies - #33156

Open
robobun wants to merge 1 commit into
mainfrom
farm/5a2216dc/peer-local-source
Open

robobun wants to merge 1 commit into
mainfrom
farm/5a2216dc/peer-local-source

Conversation

@robobun

@robobun robobun commented Jul 1, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

A peer dependency that a file: package provides is fetched from the registry anyway.

# package.json:               "plugin": "file:./vendor/plugin", "host": "file:./vendor/host"
# vendor/plugin/package.json: "peerDependencies": { "host": "*" }
$ bun install
error: ConnectionRefused downloading package manifest host        # offline
# online: the same unless the name happens to be published, in which case it is downloaded for nothing

npm and pnpm resolve this offline. The same happens for the optional-peer idiom where the consumer vendors its own copy (peerDependencies + optionalDependencies: { "host": "file:../host" }), which is the original report.

(Two other symptoms this PR originally covered landed as their own fixes while it was in review and are no longer in this diff: a required peer matching no published version made bun.lock unloadable, #26046, fixed in #38851; and a dependency plus a file: peer on the same folder wrote the same package path twice, fixed by the tree dedupe in #40564.)

Cause

get_or_put_resolved_package only accepts an existing package for a peer when the resolution kinds are comparable (npm/npm, git/git, github/github), so a folder package never satisfies an npm range and bun falls through to the registry.

Fix

find_peer_provider: after every existing acceptance path has declined, an npm-range or dist-tag peer can be bound to a same-named package that came from a non-registry source:

  • If the peer's declarer itself depends on that name, that dependency is what the declarer loads (and what the tree merges the peer onto), so the peer binds to it when it is such a package and to nothing otherwise. It is looked up regardless of --omit, since the lockfile tree contains the edge either way (Lockfile::resolution_of_dependency_named).
  • Otherwise, a tarball or git package qualifies wherever it was declared: it installs from the cache, so it works wherever the tree puts it.
  • Otherwise, a folder or link only works where its recorded path resolves, so it qualifies only as the package the root installs under the peer's name in this install (a root devDependencies folder is not a provider under --omit=dev / --production), and only when the peer's walk up the tree is certain to end at that copy: the install is not filtered (--filter can leave the root's dependencies out), it places the declarer directly under the root, nothing installs any other package under the declarer's name, and no self-contained workspace (install: self-contained workspaces for the hoisted linker (workspaces.selfContained / installConfig.hoistingLimits) #40014, a hoisting barrier) depends on the declarer (Lockfile::dedupes_onto_root_dependency). Every dependency on the declarer then dedupes onto the root's copy, one level below the provider (the tree dedupe from install: dedupe a peer bound to a folder package instead of nesting a second copy #40564).
  • That last check reads the whole dependency graph, which is still growing while peers resolve (a peer that has to come from the registry can pull in new packages). So a peer that only a root folder could still satisfy is not decided in the peer pass: find_peer_provider returns Deferred, the peer is parked instead of going to the registry, and wait_for_resolution decides the parked peers once the queue is empty and nothing is in flight (resolve_deferred_folder_peers). The check can only turn from true to false as edges arrive, so the peers it rejects go to the registry first, the rest stay parked while the graph settles, and the loop ends when a round binds everything.
  • Everything else is not used and those peers keep resolving from the registry exactly as today: an aliased fork installed under another name, a folder only a workspace member or an intermediate package depends on, a declarer the root only has under an alias or only as an omitted dependency, a declarer another version of which is installed somewhere, a declarer a self-contained workspace depends on, and the entries created for file: dependencies declared inside registry packages.

Verification

test/cli/install/bun-lock.test.ts (each asserts the exact packages keys for the host, what is and is not nested, and that the lockfile round-trips: --frozen-lockfile passes and a reinstall is a no-op; none of the hosts exist in the registry). Failing with main's src/:

  • peer bound to the root's file: dependency: one key, nothing under the consumer
  • same with the provider at file:../host outside the project
  • the original report's shape, peerDependencies plus optionalDependencies/dependencies: file: on the same name inside the plugin: one key under the plugin
  • root provides one copy and the plugin vendors another: the plugin's peer binds to its own
  • a workspace package's peer binds to the root's folder

Passing with main's src/ as well (they pin the file:-peer shapes #40564 now dedupes): dependencies plus a file: peer on one folder gives one key; two packages depending on the same folder, one of them also file:-peering on it, each keep their own copy.

test/cli/install/bun-install.test.ts, failing with main's src/:

  • registry consumer whose peer is provided by the root's file: dependency: only the consumer is requested, one key, nothing nested, round-trips
  • a root file: devDependency satisfies a registry package's peer on a normal install (no registry lookup); under --omit=dev the same peer is resolved from the registry instead
  • the original report's shape installed with --omit=optional while the root also provides a copy: the peer binds to the plugin's own vendored copy and the registry is not asked

test/cli/install/bun-install.test.ts, passing with main's src/ as well: each pins one clause of the rule above and fails when that clause is removed (in earlier revisions of this PR the last six bound the folder at a path where it does not install):

  • optionalDependencies plus a file: peer on one folder, installed with --omit=optional: one key
  • a root file: fork installed under a different name than its manifest does not satisfy a peer on the manifest name
  • a folder that only a workspace member depends on does not satisfy a peer elsewhere
  • a plugin whose own dependency on the name is a registry version outside its peer range, while the root provides a file: copy: the peer is reported unmet exactly as without the folder, and the plugin's own copy is what gets nested under it
  • a package the root only installs under an alias, and which another package nests under itself, keeps resolving its peer from the registry
  • a package the root only has as a devDependency, installed with --omit=dev so that another version takes the root slot and it gets nested: same
  • a package another version of which is installed under an intermediate, so that a second copy of it nests there: same
  • a package a self-contained workspace also depends on, so that a second copy of it sits inside that workspace's node_modules: same, and the registry copy of the peer lands inside the workspace
  • a package whose second copy only appears once another package's required peer has been fetched from the registry (nothing but that peer names the package that pulls it in): same, decided after that fetch
  • a workspace member's package under bun install --filter=<member>, which leaves the root's dependencies out of node_modules: same, the registry copy lands at the root

bun-install-registry, bun-workspaces, bun-workspaces-self-contained, bun-add, bun-add-filter, bun-lockb, isolated-install, hoist, bun-dedupe, bun-update, bun-update-transitive, catalogs, bun-pm, bun-prune, frozen-lockfile-pruned, regression/issue/40561 and the migration/ suites pass unmodified.

Related: #26046 (its lockfile symptom is fixed by #38851; the file: shapes above are this PR).

Earlier revisions

The first version deduped folder peers only at the consumer's own node and bound peers to any same-named folder entry, which review showed still placed a second copy of a root-provided host under the consumer (hidden by a test that deduplicated its lockfile matches) and merged unrelated file: stubs declared by different registry packages. Later rounds narrowed the root arm of find_peer_provider step by step (aliased providers, providers only a workspace member declares, omitted root providers, declarers that get nested for one reason or another, the incomplete graph during the peer pass) until it became the rule stated above; the constraint-pinning tests in the last list are the shapes found on the way. Three pieces this PR carried were landed independently and dropped here on rebase: the bun.lock parser tolerance for unresolved peers (#38851), the Windows path normalization for folder dependencies (#38333), and the Tree.rs change that dedupes a peer bound to a folder package against the copy an ancestor already provides (#40564), together with the entry reuse for file: peers it made unnecessary.


no test proof · iteration 19 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/cli/install/bun-lock.test.ts, test/cli/install/bun-install.test.ts

@github-actions github-actions Bot added the claude label Jul 1, 2026
@robobun

robobun commented Jul 1, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 2:44 AM PT - Aug 27th, 2026

❌ @robobun, your commit 3835677 has 3 failures in Build #106719 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 33156

That installs a local version of the PR into your bun-33156 executable, so you can run:

bun-33156 --bun

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Found 6 issues this PR may fix:

  1. DependencyLoop crash #5789 - DependencyLoop crash when installing git packages whose peer dependencies trigger a spurious registry lookup
  2. DependencyLoop with github dependencies #8031 - DependencyLoop when adding a second version of a GitHub dependency (git-resolved peer deps hit the registry)
  3. DependencyLoop Error when trying to install a specific tag of a Github repo that's already installed #21405 - DependencyLoop when reinstalling a GitHub repo at a different tag (same peer-dep-hits-registry mechanism)
  4. crash: bun add remote package tar.gz dependency loop and errors #20647 - DependencyLoop crash when adding a remote tarball package whose peer dependencies cause a registry lookup
  5. bun.lock failed on file:../xxx #17060 - bun.lock fails to parse on second install with file: dependencies due to malformed/duplicate lockfile entries
  6. error: Error loading lockfile: InvalidPackageInfo #26046 - Error loading lockfile: InvalidPackageInfo after installing packages with unresolved peer dependencies

If this is helpful, copy the block below into the PR description to auto-close these issues on merge.

Fixes #5789
Fixes #8031
Fixes #21405
Fixes #20647
Fixes #17060
Fixes #26046

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Jul 1, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Peer resolution now accepts non-registry package sources, folder packages can be reused, unresolved peer ranges remain loadable in lockfiles, and install tests cover the updated behavior.

Changes

Peer Dependency and Folder Resolution Handling

Layer / File(s) Summary
Peer satisfaction checks
src/install/PackageManager/PackageManagerEnqueue.rs
Adds resolution_provides_peer and uses it in both peer-matching paths so npm/dist-tag peers can be satisfied by non-registry resolutions.
Folder package reuse and serialization
src/install/PackageManager/PackageManagerEnqueue.rs, src/install/lockfile/Tree.rs, src/install/lockfile/Package.rs
Reuses existing folder package ids, deduplicates folder-tagged hoisting, and updates folder path serialization in lockfiles.
Unresolved peer lockfile handling
src/install/lockfile/bun.lock.rs
Allows unresolved peer dependencies to continue through lockfile parsing alongside optional dependencies.
Install and lockfile regressions
test/cli/install/bun-lock.test.ts, test/cli/install/bun-install.test.ts, test/cli/install/bun-install-registry.test.ts, test/cli/install/bun-add.test.ts
Adds regression tests for local file: peers, optional-peer patterns, unresolved peer ranges, and updates expected install counts and path normalization.
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes address #26046 by allowing unresolved peers to remain in the lockfile and by keeping bun pm ls/frozen installs working.
Out of Scope Changes check ✅ Passed The added changes stay within the install/lockfile peer-resolution bugfix area and do not appear unrelated to the linked issue.
Title check ✅ Passed The title clearly and concisely describes the primary change: resolving peers provided by file: dependencies.
Description check ✅ Passed The description explains the problem, cause, fix, scope, limitations, and verification results. It uses equivalent headings and provides more detail than the required template sections.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/install/PackageManager/PackageManagerEnqueue.rs`:
- Around line 2311-2323: The fallback in PackageManagerEnqueue’s
`PackageIndexEntry::Ids` handling is running too early, so `list[0]` can be
returned before `resolution_provides_peer` has a chance to match a valid peer
provider. Update the peer-resolution flow around the `resolution_provides_peer`
scan and the `success_fn`/`ResolvedPackageResult` return so the helper is
checked before any fallback to the first registry entry, ensuring `file:`,
`git:`, or tarball providers can win when they satisfy the peer.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 57d1b9fa-b460-4a78-8aac-9adbe12cb0d0

📥 Commits

Reviewing files that changed from the base of the PR and between 52a1ddf and 44883f4.

📒 Files selected for processing (3)
  • src/install/PackageManager/PackageManagerEnqueue.rs
  • src/install/lockfile/Tree.rs
  • test/cli/install/bun-lock.test.ts

Comment thread src/install/PackageManager/PackageManagerEnqueue.rs

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't find any bugs, but this changes core peer-resolution semantics (accepting any non-registry resolution as satisfying an npm peer range) and reroutes folder placement through hoist_dependency, so it's worth a human look at the install/lockfile behavior.

Extended reasoning...

Overview

This PR touches three files: src/install/PackageManager/PackageManagerEnqueue.rs (adds resolution_provides_peer and calls it in two places inside get_or_put_resolved_package; adds a dedup-by-resolution check before appending a transitive folder package), src/install/lockfile/Tree.rs (routes Folder-resolved packages through hoist_dependency with the current node as its own hoist root instead of placing directly), and two new tests in test/cli/install/bun-lock.test.ts.

Security risks

None apparent. The change does not touch auth, network, or path handling; it only affects which already-resolved package entry a peer dependency binds to and how folder packages are deduped in the tree.

Level of scrutiny

High. This is core bun install resolution and lockfile-tree logic that runs on every install. The new resolution_provides_peer helper introduces a semantic rule — any Folder/Symlink/tarball/Git/Github resolution satisfies any npm-range or dist-tag peer, without version comparison — which is a deliberate policy choice (matching npm/pnpm behavior per the description) rather than a mechanical fix. The Tree.rs change replaces an unconditional Placement with a call into hoist_dependency, which now exposes folder packages to the Resolve/ResolveReplace/ResolveLater/DependencyLoop paths at the same node; the next_id == hoist_root_id argument should keep them from hoisting to a parent, but the additional code paths deserve a maintainer's eye.

Other factors

The PR is well-described with clear repros, root-cause analysis, and two focused tests that exercise offline resolution, --frozen-lockfile round-tripping, and second-install idempotence. The author reports the broader install/lock/workspace/peer suites are unchanged. The bug-hunting pass found nothing. Still, given how many edge cases peer resolution has (optional peers, workspace-provided peers, git peers, the "incorrect peer dependency" warn-and-use fallback ordering), I'd rather a human confirm the new acceptance rule and the folder→hoist_dependency reroute are the intended shape before this lands.

@robobun

robobun commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator Author

I checked all six against this branch.

#26046: fixed, and now part of this PR. Reproduced on main with the issue's exact steps: rolldown-plugin-solid-oxc has a non-optional peer on solid-jsx-oxc@*, which only has a prerelease, so the peer resolves to nothing. The installer tolerates that and writes the lockfile, then bun pm ls and the next install reject it with Failed to resolve peer dependency 'solid-jsx-oxc' for package 'rolldown-plugin-solid-oxc' / failed to parse lockfile. d449068 makes the parser leave an unresolvable peer unresolved, matching what the installer that wrote the file did, and adds a regression test. bun pm ls now prints the tree on the issue's repro.

The other five are different mechanisms and are not fixed here:

So only Fixes #26046 applies; it's in the PR description now.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
src/install/lockfile/bun.lock.rs (2)

2818-2835: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Root-dependency peer unresolved handling not updated to match the new fix.

This loop resolving the root package's dependencies still only tolerates Behavior::OPTIONAL on lookup miss; it doesn't skip for dep.behavior.is_peer() the way the fix at line 2971 now does for nested package dependencies. If the root package.json declares a peer dependency that Bun already tolerated (unresolved) at install time, loading that lockfile will still hit dependency_resolution_failure and fail to parse here — the exact symptom this PR is fixing, just at the root level instead of nested packages.

🐛 Proposed fix
                 let Some(&res_id) = pkg_map.get(dep.name.slice(string_buf)) else {
-                    if dep.behavior.contains(Behavior::OPTIONAL) {
+                    if dep.behavior.contains(Behavior::OPTIONAL) || dep.behavior.is_peer() {
                         continue;
                     }

Since verify_resolutions in PackageManagerResolution.rs already treats unresolved peers this way (failed_dep.behavior.is_peer()), this appears to be an existing, established tolerance pattern that this loop is simply missing.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/install/lockfile/bun.lock.rs` around lines 2818 - 2835, The root
dependency resolution loop still only skips unresolved OPTIONAL deps, so it must
be updated to also tolerate unresolved peer deps like the nested-package fix
does. In the dependency lookup miss branch in the root package dependency walk,
add the same dep.behavior.is_peer() check used elsewhere (for example in
verify_resolutions and the newer nested dependency handling) before calling
dependency_resolution_failure, so unresolved peer dependencies are skipped
instead of failing lockfile parsing.

2894-2910: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Same gap for workspace-package dependency resolution.

This workspace-dependency loop has the identical pattern as the root-dependency loop above: it only skips dependency_resolution_failure for Behavior::OPTIONAL, not for is_peer(). A workspace package declaring an unresolved peer dependency (which the installer already tolerates at install time per the PR's stated behavior) would still fail to parse back from the lockfile.

🐛 Proposed fix
                     else {
-                        if dep.behavior.contains(Behavior::OPTIONAL) {
+                        if dep.behavior.contains(Behavior::OPTIONAL) || dep.behavior.is_peer() {
                             continue;
                         }

This is the same bug class fixed at line 2971 for the general package dependency loop; per the "fix the whole class in the same PR" guideline, sibling sites sharing this pattern should be updated together.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/install/lockfile/bun.lock.rs` around lines 2894 - 2910, The workspace
dependency resolution path in the lockfile parser still rejects unresolved peer
dependencies because it only skips failure for Behavior::OPTIONAL. Update the
dependency loop that uses pkg_map and dependency_resolution_failure to also
bypass the error when dep.is_peer() is true, matching the fix already applied in
the general package dependency loop. Keep the behavior consistent for workspace
packages so unresolved peers round-trip without ParseError::InvalidPackageInfo.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@src/install/lockfile/bun.lock.rs`:
- Around line 2818-2835: The root dependency resolution loop still only skips
unresolved OPTIONAL deps, so it must be updated to also tolerate unresolved peer
deps like the nested-package fix does. In the dependency lookup miss branch in
the root package dependency walk, add the same dep.behavior.is_peer() check used
elsewhere (for example in verify_resolutions and the newer nested dependency
handling) before calling dependency_resolution_failure, so unresolved peer
dependencies are skipped instead of failing lockfile parsing.
- Around line 2894-2910: The workspace dependency resolution path in the
lockfile parser still rejects unresolved peer dependencies because it only skips
failure for Behavior::OPTIONAL. Update the dependency loop that uses pkg_map and
dependency_resolution_failure to also bypass the error when dep.is_peer() is
true, matching the fix already applied in the general package dependency loop.
Keep the behavior consistent for workspace packages so unresolved peers
round-trip without ParseError::InvalidPackageInfo.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: c097e87a-ce0e-4116-b958-2acfac4f697f

📥 Commits

Reviewing files that changed from the base of the PR and between 44883f4 and d449068.

📒 Files selected for processing (2)
  • src/install/lockfile/bun.lock.rs
  • test/cli/install/bun-install.test.ts

Comment thread src/install/lockfile/bun.lock.rs Outdated
Comment thread test/cli/install/bun-lock.test.ts Outdated
@robobun

robobun commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator Author

8a00881 addresses the review feedback and the Windows CI failure from build 67393:

  • The unresolved-peer tolerance now lives in one helper used by all three dependency-resolution loops in the bun.lock parser (root, workspace, package), since the root and workspace shapes are reachable too (CodeRabbit's outside-diff comments and the bun.lock.rs thread). New round-trip test covering both.
  • bun-lock.test.ts failed on Windows because the Folder arm of Package::parse_dependency stores the platform-native relative path, unlike the Workspace arm next to it, so transitive file: resolutions were written to bun.lock with backslashes (which bun.lockb.rs debug-asserts against). Fixed at the source rather than loosening the assertion.
  • Deduping transitive folder packages changes N packages installed by one in three transitive file dependencies tests (the same folder was previously two identical package entries); those expectations are updated. The on-disk trees they assert are unchanged.

Locally: bun-lock 16/16, bun-install-registry 223 pass with only the two git+ssh network tests failing (they need outbound ssh), isolated-install 48/48, bun-workspaces 63/63.

@robobun
robobun force-pushed the farm/5a2216dc/peer-local-source branch 2 times, most recently from 19286ae to 42f5599 Compare July 1, 2026 02:09

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/cli/install/bun-install.test.ts`:
- Around line 1478-1481: The test in bun-install.test.ts only verifies that one
unresolved peer entry exists, so it does not prove both root and workspace peers
survived the round-trip. Tighten the assertions around the bun.lock contents by
checking for both unresolved peer records using the existing lockfile read in
the test, and keep the negative check for any resolved "bar@" entry so the test
fails if either peer entry is dropped. Use the surrounding install test case and
the lockfile assertions near the current expect(lockfile) calls to locate the
update.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 7a3fa23b-d6e9-4735-8135-78598954723d

📥 Commits

Reviewing files that changed from the base of the PR and between 8a00881 and 42f5599.

📒 Files selected for processing (5)
  • src/install/lockfile/Package.rs
  • src/install/lockfile/bun.lock.rs
  • test/cli/install/bun-add.test.ts
  • test/cli/install/bun-install-registry.test.ts
  • test/cli/install/bun-install.test.ts

Comment thread test/cli/install/bun-install.test.ts Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No new issues found after the fixes in 8a00881/42f5599c — but this changes peer-resolution semantics, folder-package deduplication, and lockfile-parse tolerance across several coupled sites, plus updates existing test expectations, so it warrants a human pass (and CI build #67420 still shows musl build failures and Windows bun-add.test.ts failures).

Extended reasoning...

Overview

This PR touches four distinct pieces of the installer:

  • PackageManagerEnqueue.rs: new resolution_provides_peer helper accepting non-registry resolutions (folder/symlink/tarball/git/github) as peer providers, plus dedup of transitive folder packages by (name, resolution) before appending.
  • Tree.rs: folder-resolved packages now route through hoist_dependency (with next_id as the hoist root) instead of direct placement, so same-node dedup runs.
  • bun.lock.rs: three dependency-resolution loops (root, workspace, package) now tolerate unresolved peers via a shared dependency_can_be_unresolved helper.
  • Package.rs: Windows-only posix normalization of the relative folder path stored for transitive file: deps.

Plus four new tests and updates to nine existing test expectations (N packages installed counts in bun-install-registry.test.ts and a Windows path assertion in bun-add.test.ts).

Security risks

None identified. No auth, crypto, network trust, or path-traversal surface is touched; the changes are confined to internal resolution/hoisting bookkeeping and lockfile round-tripping.

Level of scrutiny

High. This is core package-manager resolution logic where subtle ordering matters — e.g., resolution_provides_peer deliberately runs after the existing "incorrect peer dependency" fallback, and the Tree.rs change replaces an unconditional placement with a call that can now return Hoisted/Resolve/ResolveReplace/etc. The folder-dedup in enqueue_dependency changes when a new package entry is created vs. reused, which is why existing package-count assertions moved. Each piece looks correct in isolation and is well-tested, but the interaction surface (peer + folder + hoisting + lockfile round-trip) is exactly where installer regressions hide.

Other factors

  • My two prior inline comments (unresolved-peer tolerance at only 1 of 3 parse sites; Windows backslash in folder resolutions) were both addressed with code fixes and new tests, and the threads are resolved.
  • The robobun CI status for the latest reported build (#67420) shows musl build failures and Windows bun-add.test.ts failures — worth confirming those are green on the current head before merge.
  • Existing test expectations were updated (package counts 14→13, 13→12); the PR description explains why, and the on-disk-tree assertions in those tests are unchanged, but a maintainer should confirm the count change is expected and not masking a dropped install.

@dylan-conway

Copy link
Copy Markdown
Member

@robobun are you sure this is correct? peers are required and only optional when they have optional: true in peerDependenciesMeta right? (looking at the changes in bun.lock.rs)

@robobun

robobun commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator Author

Right, and that part is unchanged: a peer is only optional via peerDependenciesMeta.optional (serialized as optionalPeers), and this PR never marks anything else optional. Required peers still resolve and auto-install exactly as before.

The bun.lock.rs change is only about what the parser does with a lockfile in which a required peer recorded no resolution. The installer already allows that state on purpose: verify_resolutions skips unresolved peers before deciding whether the install failed (PackageManagerResolution.rs:370, the TODO lockfile rewrite: remove this and make non-optional peer dependencies error if they did not resolve. Need to keep this for now because old lockfiles might have a peer dependency without the optional flag set comment), so bun install exits 0 and writes the lockfile, and loading the same state from a binary lockfile works. Only the text parser treated it as a parse error, so bun rejects a file it just wrote:

bun init -y
bun add -d rolldown-plugin-solid-oxc   # its peer "solid-jsx-oxc": "*" only has a prerelease published

bun pm ls
error: Error loading lockfile: InvalidPackageInfo

bun install --frozen-lockfile
error: Failed to resolve peer dependency 'solid-jsx-oxc' for package 'rolldown-plugin-solid-oxc'
InvalidPackageInfo: failed to parse lockfile: 'bun.lock'
error: lockfile had changes, but lockfile is frozen

That is #26046, and the same thing happens for any dependency carrying a peer range that matches nothing published ("peerDependencies": { "a-dep": ">=99.0.0" }), including on the root or a workspace package.

So the parser now tolerates exactly what the resolver and the writer already tolerate, nothing more. If you would rather a required peer that resolves to nothing fail the install itself (what that TODO describes), I can do that instead or as a follow-up, but it is a behavior change (the bun add above becomes an error), and the parser still has to accept the lockfiles existing bun versions have already written.

@robobun
robobun force-pushed the farm/5a2216dc/peer-local-source branch 2 times, most recently from 290c7af to 0c54917 Compare July 1, 2026 09:20

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/install/lockfile/Package.rs`:
- Around line 1820-1839: The Windows path append logic in the folder assignment
block is duplicated from the `Tag::Workspace | Folder | Symlink | Tarball`
handling in `parse_dependency`, so extract that inline/non-inline POSIX
normalization into a shared helper. Add a helper near
`Package`/`parse_dependency` (for example one that takes `StringBuilder` and
`folder_path` and returns the appended `String`) and update both call sites to
use it, keeping the existing `String::can_inline` and
`dangerously_convert_path_to_posix_in_place` behavior unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: c01e5c17-b59c-4aa6-b95e-e074d8c79197

📥 Commits

Reviewing files that changed from the base of the PR and between 42f5599 and 0c54917.

📒 Files selected for processing (8)
  • src/install/PackageManager/PackageManagerEnqueue.rs
  • src/install/lockfile/Package.rs
  • src/install/lockfile/Tree.rs
  • src/install/lockfile/bun.lock.rs
  • test/cli/install/bun-add.test.ts
  • test/cli/install/bun-install-registry.test.ts
  • test/cli/install/bun-install.test.ts
  • test/cli/install/bun-lock.test.ts

Comment thread src/install/lockfile/Package.rs Outdated
@robobun

robobun commented Jul 1, 2026 •

Copy link
Copy Markdown
Collaborator Author

CI status for the current head (6ce9858, rebased on main): 282 jobs pass, 2 fail, both unrelated to this diff.

  • darwin 26 aarch64 - test-bun: buildkite-agent artifact download timed out after 120s for step 'darwin-aarch64-build-bun'. The job exits before running a single test. That agent pool has failed this way on every build of this PR (67393, 67420, 67426, 67493, 67535, 68610), independent of the base commit.
  • darwin 14 x64 - test-bun: test/js/bun/terminal/terminal.test.ts → "Bun.spawn with terminal option > creates subprocess with terminal attached" hit the 90s timeout on that runner (88 of the file's 90 tests pass there). A PTY spawn test; this PR only touches src/install.

The test-net-connect-memleak.js failures reported earlier are gone on this run, consistent with it having been a transient CI-wide issue rather than anything in this branch.

Every install suite is green on this head, in CI and locally: bun-lock 15/15, bun-install-registry 223 pass (only the two git+ssh network tests fail, they need outbound ssh), bun-install 169 pass with all failures in the pre-existing network set, isolated-install 55/55, bun-workspaces 63/63, bun-lockb 6/6.

@robobun

robobun commented Jul 1, 2026 •

Copy link
Copy Markdown
Collaborator Author

Another trigger for the duplicate packages key this PR fixes in Tree::process_subtree, and a more common shape than the peer ones covered here: the root package listing the same file: folder in both dependencies and devDependencies.

// package.json
{ "dependencies":    { "a": "file:./vendor/a" },
  "devDependencies": { "a": "file:./vendor/a" } }
$ bun install                   # exit 0, but bun.lock now has two "a" keys under "packages"
$ bun install --frozen-lockfile
error: Duplicate package path
InvalidPackageKey: failed to parse lockfile: 'bun.lock'
error: lockfile had changes, but lockfile is frozen

The Tree.rs change here already fixes it. I put two tests for this shape in test/cli/install/bun-lock.test.ts on farm/e4608f43/lockfile-dup-folder-dep; both fail on main and pass on this branch at 0c54917 without any src changes, so they are worth folding in. They reach two branches of hoist_dependency that the peer-shaped tests do not for a Folder resolution:

  • the same file: folder in dependencies and devDependencies: one "a" entry, and --frozen-lockfile round-trips (the res_id == package_id merge)
  • two different file: folders under the same name across dependencies and devDependencies: one entry wins, matching the existing local-tarball behavior (the is_dev() mismatch path)

Not opening a separate PR since the src fix is identical.

@robobun
robobun force-pushed the farm/5a2216dc/peer-local-source branch from 0c54917 to 6ce9858 Compare July 5, 2026 17:55
@robobun

robobun commented Jul 5, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto main (6ce9858) to clear the conflict with #32182, which rewrote the three dependency-resolution loops in bun.lock.rs to try a deferred-peer resolution before the package-map lookup.

Resolution: kept #32182's new is_deferred_peer / resolve_peer_dep_version_based wrapper and its row.key_loc at all three sites, and reapplied only this PR's change inside each unresolvable branch, i.e. dependency_can_be_unresolved(dep) (optional or peer) in place of the Behavior::OPTIONAL-only check. No other part of either change was altered; a peer that #32182 now resolves version-based never reaches the tolerance branch.

Verified on the rebase: bun-lock 15/15, bun-install-registry 223 pass (only the two git+ssh network tests fail), bun-install 169 pass with all failures in the pre-existing network set, isolated-install 55/55, bun-workspaces 63/63, bun-lockb 6/6, and the original repro from the description still installs offline, writes one plugin/host entry, and passes --frozen-lockfile.

@robobun

robobun commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator Author

Still reproduces on 1.4.0-canary.1 (b66764f): the first test in this PR (peer provided by a root file: dependency) fails against that build with error: GET http://localhost:<port>/zz-host-a - 404. Another report of the same shape came in independently, seen with the isolated linker as well, where the failed install also leaves a partial node_modules/.bun store entry behind.

@robobun

robobun commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator Author

While working on #37289 (the migration half of the "install then --frozen-lockfile fails on an unchanged tree" problem) I independently reproduced two more user-visible failures that the Tree.rs dedupe hunk in this PR fixes, so adding the evidence here.

Shape, no foreign lockfile involved:

mkdir -p vendor/gamma vendor/delta packages/ws-a
cat > vendor/gamma/package.json <<'EOF'
{"name":"gamma","version":"1.0.0","peerDependencies":{"delta":"*"},"peerDependenciesMeta":{"delta":{"optional":true}},"optionalDependencies":{"delta":"file:../delta"}}
EOF
echo '{"name":"delta","version":"2.0.0"}' > vendor/delta/package.json
echo '{"name":"ws-a","version":"0.1.0","dependencies":{"delta":"file:../../vendor/delta"}}' > packages/ws-a/package.json
echo '{"name":"sandbox","version":"1.0.0","workspaces":["packages/ws-a"],"dependencies":{"gamma":"file:vendor/gamma"}}' > package.json

bun install && bun install --frozen-lockfile

On current main (and in a 60-iteration loop):

  1. bun install && bun install --frozen-lockfile fails 52/60 with error: lockfile had changes, but lockfile is frozen, even though a plain re-install writes a byte-identical bun.lock. The text lockfile parser binds the optional-peer edge to the recorded "gamma/delta" placement, then the tree rebuild sends the folder dependency through the Tag::Folder shortcut, which skips hoist_dependency's same-name scan and places a second entry for the same node_modules folder. Cleaning rebuilds the tree with the peer slot re-derived (one entry), so the frozen check's structural Lockfile::eql compares 2 entries against 1 and reports a phantom change. The serialized text never shows the duplicate, which is why the lockfile on disk looks stable while frozen installs fail.
  2. The remaining 8/60 iterations fail the first bun install outright with EEXIST: File exists: failed to link package: delta@vendor/delta (link): the duplicate node entry produces two install tasks racing to link the same folder.

With the hoist_dependency::<true, METHOD>(next_id, next_id, ...) change from this PR applied, the same loop is 0/40 failures and both symptoms disappear. So besides the duplicate-package-path parse break described in the PR body, this hunk also fixes a --frozen-lockfile false positive on unchanged trees and an intermittent install failure. Both only became visible after bbe3f6a (#35681) started re-deriving optional-peer slots during cleaning, which made the pre-clean (duplicated) and post-clean (deduplicated) trees diverge; before that both sides were equally duplicated and eql never noticed.

A regression test for this shape, in case it is useful here: install once, then assert --frozen-lockfile exits 0 and a re-install leaves bun.lock byte-identical (it fails nearly every run on main without the Tree.rs hunk). I have it written against test/cli/install/bun-lock.test.ts and can post it if wanted.

@alii alii left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes. The parser change and the posix folder path are fine. The tree half is not: every Folder-resolved edge is still placed under the package that depends on it, and the new peer binding now routes peers through that path.

  • Tree.rs: a peer bound to a root file: package gets a second placement under the consumer. The PR's own root-provides case installs the host twice, an npm consumer gets a placement that never installs, and a root provider at file:../host makes install exit 1. The last two work on main.
  • resolution_provides_peer binds by name to any Folder entry, including the stubs created for file: deps declared inside npm packages.
  • The new bun-lock test dedupes its lockfile matches with new Set, which is what hides the extra placement.

Comment thread src/install/lockfile/Tree.rs Outdated
Comment thread src/install/PackageManager/PackageManagerEnqueue.rs Outdated
Comment thread test/cli/install/bun-lock.test.ts Outdated
@robobun
robobun force-pushed the farm/5a2216dc/peer-local-source branch from 6ce9858 to ed8a997 Compare August 13, 2026 02:38
Comment thread src/install/PackageManager/PackageManagerEnqueue.rs Outdated
Comment thread src/install/lockfile.rs Outdated
@robobun
robobun force-pushed the farm/5a2216dc/peer-local-source branch from 00f1fda to d5eb5e9 Compare August 15, 2026 07:41
@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

@alii and one more small one, d5eb5e9: the file: peer reuse was applying the install's --omit filter when looking up the declarer's own edge, but that entry exists and is in the lockfile regardless of omits, so under --omit=optional an optionalDependencies + file: peer pair got two entries and the dependency-loop error. The filter is now optional on the helper and only find_peer_provider uses it. One test. Nothing else changed; suites green locally.

Comment thread src/install/lockfile.rs Outdated
Comment thread src/install/PackageManager/PackageManagerEnqueue.rs Outdated
@robobun
robobun force-pushed the farm/5a2216dc/peer-local-source branch from d5eb5e9 to 99fc3c4 Compare August 15, 2026 10:01
@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

@alii one more narrowing of the provider rule since your approval, now at 99fc3c4 (single commit, rebased on current main). Two more holes were found in the folder arm of find_peer_provider, both of the same kind as before (binding a folder the tree then cannot dedupe the peer onto, which ends in a dependency loop exit):

  • The declarer's own same-named dependency was looked up through the install's --omit filter, so under --omit=optional the optional-peer idiom bound to the root's copy instead of the plugin's own and the lockfile tree rejected it. The own edge is now looked up unfiltered, and when the declarer has one it is the only thing the peer can bind to (it is what the declarer loads; anything else collides with it at the declarer's node). If it is a registry package outside the peer's range, the peer is left alone exactly as on main.
  • "The declarer sits directly under the root" was checked as "the root has some edge to it", which an aliased root edge satisfied while a transitive edge under the real name still nested a second copy. It is now Lockfile::dedupes_onto_root_dependency: every edge to the declarer uses one name and the root holds one of them.

Each clause has a test in bun-install.test.ts that fails on the previous head (the --omit=optional and out-of-range shapes exited 1 with dependency loop; the alias shape bound the folder). The rest of the diff is unchanged from what you approved; the PR description's Fix and Verification sections are updated. Locally: bun-install, bun-lock, bun-install-registry, bun-workspaces, bun-add, bun-lockb, isolated-install, hoist and pnpm-lock-v9 all pass; a fresh CI run is going on this head.

@robobun
robobun force-pushed the farm/5a2216dc/peer-local-source branch from 99fc3c4 to 2d9c749 Compare August 15, 2026 10:45
Comment thread src/install/lockfile.rs Outdated
@robobun
robobun force-pushed the farm/5a2216dc/peer-local-source branch from 2d9c749 to c172f8f Compare August 15, 2026 11:06
@robobun

robobun commented Aug 15, 2026 •

Copy link
Copy Markdown
Collaborator Author

@alii rebased again, now c172f8f on current main (one commit, mergeable). What changed on the way, beyond the conflicts:

The description's Fix and Verification sections describe the PR as it stands now. CI on this head (build 97943): 177 jobs passed, no test failures; the two ":darwin: 14 aarch64 - test-bun" shards expired after waiting an hour for an agent, which is hitting about a third of today's builds on that lane. I have asked for those two jobs to be retried rather than pushing again.

All review threads are resolved (your round-two ones were addressed before your later sign-off; I resolved them now so the list is clean). GitHub still shows your request-changes review from the 13th as the standing one, so a fresh approval is what it needs once you are happy with the final rule.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed c172f8f and the automated bug-hunting pass found no issues. Since the find_peer_provider / dedupes_onto_root_dependency rule has been narrowed several times after the earlier human approval, a fresh look at the final shape of that rule would be worthwhile before merge.

What was checked on this head:

  • The two dedupes_onto_root_dependency gaps flagged on the previous head — the missing is_enabled(root_features) on the root edge, and the same-name/different-package blocker at an intermediate — are both addressed in the current diff, with matching tests in bun-install.test.ts.
  • A candidate concern that dedupes_onto_root_dependency reads incomplete resolution state during concurrent enqueue was examined and ruled out.
  • The Tree.rs Folder-peer walk-up now aligns with the invariant dedupes_onto_root_dependency enforces (single install name, root places it, nothing else under that name).
Extended reasoning...

Overview

This PR changes peer-dependency resolution and tree hoisting in bun install so that a peer satisfied by a file: (folder/link) package binds to it instead of hitting the registry, and so that two edges of one package pointing at the same folder produce a single lockfile key. It touches PackageManagerEnqueue.rs (new find_peer_provider fallback and a file:-peer reuse block), lockfile.rs (three new helpers: declaring_package, resolution_of_dependency_named, dedupes_onto_root_dependency), Tree.rs (Folder edges now go through hoist_dependency with peers walking up to the hoist root and non-peers clamped to their own node), plus ~900 lines of tests across bun-install/bun-lock and one inverted expectation in the pnpm migration suite.

Security risks

None identified. The change is confined to how already-resolved local packages are matched against peer edges and where they land in the tree; it does not touch network fetching, tarball extraction, path validation, or credential handling. The only user-controlled input newly consulted is the dependency name hash and behavior bits already parsed from package.json.

Level of scrutiny

High. This is core install correctness logic — the exact path that determines which package satisfies a peer and where it is placed. The PR's own review history (14+ iterations, at least eight distinct correctness gaps found and fixed in the folder-provider rule alone: aliased providers, workspace-only providers, omitted root providers, nested declarers, aliased declarers, filtered sibling edges, filtered root declarer edges, same-name blockers) shows how sensitive the invariant is: find_peer_provider decides at enqueue time that the tree builder will later dedupe the peer onto a specific placement, and every case where that prediction is wrong either writes an uninstallable folder path or errors. That coupling between two phases is inherently fragile and warrants a maintainer's eyes on the final rule.

Other factors

  • alii approved an earlier revision (ef3a32b), but the find_peer_provider folder arm and the entirety of dedupes_onto_root_dependency have been rewritten several times since. robobun has flagged each change to alii, so a re-review is already expected.
  • No bugs were found on c172f8f. My previous findings on the prior head are verifiably addressed in the diff (the root_features gate and the else if dep.name_hash == install_name && resolutions[dep_id] != invalid_package_id clause are both present), and one candidate about reading resolutions during concurrent enqueue was ruled out by the verifier.
  • Test coverage is thorough: each clause of the provider rule has a positive and a negative test, and every lockfile-writing test round-trips through --frozen-lockfile. The one changed existing expectation (pnpm-lock-v9) is explained and looks correct — the redundant with/has-peer/peer copy is now deduped onto with/peer.
  • The tarball/git arm of find_peer_provider (accepted anywhere, no range check) was discussed and intentionally kept as-is for a follow-up; that is a design call the maintainer should confirm.

Jarred-Sumner pushed a commit that referenced this pull request Aug 21, 2026
### Problem
- `bun pm ls` without `--all` prints a root dependency once per entry
the root has for it. A workspace `a` that is also in the root's
`devDependencies` prints twice under a `node_modules (1)` header. So
does a name declared in two groups (`typescript` in #8283's example).
- Cause: `src/runtime/cli/package_manager_command.rs:644` prints the
root's dependency list as is: one entry per declaration plus one per
workspace member. `node_modules` has one folder per name.

### Fix
- Stable sort the root's entries by name and drop the later entries of a
name (`dedup_by`). `--trusted` filters the result.
- Correct because one name is one `node_modules/<name>` folder, the same
collapse `Tree::hoist_dependency` applies. The list is in the tree
builder's order, so the entry kept is the one the tree kept. An alias
has its own name and keeps its row. The listing does not read the stored
tree, so an older `bun.lockb` prints as before (Notes).
- Verified: `test/cli/install/bun-pm.test.ts`. Three new tests
(bun.lock, bun.lockb, `--trusted`) fail on the released bun. One more
pins that a root optional peer bound to a dependency's copy stays
listed. On this repo's lockfile the only change is the removed
`@types/bun` row.

### Background
- Each package owns one range of `buffers.dependencies`. The root's
range holds its declared dependencies plus one `Behavior::WORKSPACE`
entry per workspace member, sorted by group and name. Entries of one
name share one folder.
- The hoisted tree (`buffers.trees`, `buffers.hoisted_dependencies`)
models `node_modules`: one tree per folder, one entry per name. `--all`
prints it and the header `(N)` counts its entries. The default listing
prints only the root's range.

<details><summary>Notes</summary>

Repro without a registry:

```sh
mkdir -p d/packages/a && cd d
echo '{"name":"root","workspaces":["packages/*"],"devDependencies":{"a":"workspace:*"}}' > package.json
echo '{"name":"a","version":"1.0.0"}' > packages/a/package.json
bun install && bun pm ls
```

Released bun prints `a@workspace:packages/a` on two lines, three when
`a` is also in `dependencies`. `bun pm ls --all` prints it once. `bun pm
ls --trusted` with `trustedDependencies: ["a"]` prints it twice. In this
repo `bun pm ls` prints `@types/bun@workspace:packages/@types/bun`
twice.

- The first version of this PR printed the tree's root folder filtered
to the root's id range. A self-review found a case it drops: a root
optional peer that a dependency provides. When the tree builder binds
that peer late, it stores the folder entry under the provider's
dependency id (`Tree.rs`, `ResolveReplace`). bun 1.4.0 re-hoists on load
and on a second pass stores the root's own id, but a `bun.lockb` written
by bun 1.3.x keeps the provider's id, and bun does not rewrite a
lockfile that needs no changes. Checked with a `bun.lockb` written by
bun 1.3.14 from the `optional-peer-hoist-provider` and `-target`
registry fixtures: the tree version listed only the provider, the
current version lists both, as 1.3.14 and the released 1.4.0 do. The new
optional peer test covers the fresh lockfile shape of this case.
- Dedupe by package id was rejected: `bar` and `bar-alias: npm:bar`
share one package id but are two folders. The test keeps both rows.
- The `--trusted` test uses bun.lock only. bun.lockb stores only hashes
of `trustedDependencies`, and `has_trusted_dependency` does not trust a
hash alone since #31339, so `--trusted` lists nothing from a bun.lockb.
Released bun does the same. Not related to this change.
- A real conflict (workspace `a` plus `"a": "file:..."` at the root)
does not reach `pm ls` today: `bun install` writes a duplicate package
path and the lockfile fails to load (the area of #33156).
- #8283 asks for grouped output and `--json`. This PR removes the
duplicate row in its example and nothing else, so it does not close that
issue.
- Open PRs that touch this branch and do not fix this: #38923 (filter by
what is on disk), #38952 (header wording). The fold branch of #39403 has
the same loop.
- Suites run with the debug build: `test/cli/install/bun-pm.test.ts` (22
pass), `bun-install-lifecycle-scripts.test.ts -t trusted` (55 pass),
`bun-install-registry.test.ts -t "migration is out of sync"`,
`test/regression/issue/24502`. `cargo fmt --check` is clean. `bun pm ls
--all` on this repo is unchanged byte for byte.
</details>

<!-- robobun:evidence:begin -->

---

**[review]** gate passed · iteration 0 · 2 files touched

<details><summary>fails on main (without fix)</summary>

```console
ASAN without fix: 3 FAILED
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/cli/install/bun-pm.test.ts
bun test v1.4.0 (6e906e4)

test/cli/install/bun-pm.test.ts:
(pass) should list top-level dependency [386.82ms]
(pass) should list all dependencies [333.90ms]
(pass) should list top-level aliased dependency [329.02ms]
(pass) should list aliased dependencies [346.24ms]
(pass) should list only trusted dependencies with --trusted [464.07ms]
(pass) should list only trusted dependencies with --all --trusted [339.68ms]
(pass) should list trusted transitive dependencies under untrusted parents with --all --trusted (isolated) [356.01ms]
(pass) should list nothing with --trusted when no dependencies are trusted [346.58ms]
597 |   expect(await exists(join(package_dir, lockfile))).toBeTrue();
598 |   urls.length = 0;
599 | 
600 |   const [stdout, stderr, exitCode] = await spawnAndCollect("pm", "ls");
601 |   expect(stderr).toBe("");
602 |   expect(normalizeBunSnapshot(stdout, package_dir)).toMatchInlineSnapshot(`
                                                          ^
error: expect(received).toMatchInlineSnaps
... (truncated)

release without fix: 3 FAILED
bun test v1.4.0-canary.1 (6e906e4)

test/cli/install/bun-pm.test.ts:
(pass) should list top-level dependency [17.59ms]
(pass) should list all dependencies [15.04ms]
(pass) should list top-level aliased dependency [18.13ms]
(pass) should list aliased dependencies [26.89ms]
(pass) should list only trusted dependencies with --trusted [29.53ms]
(pass) should list only trusted dependencies with --all --trusted [30.36ms]
(pass) should list trusted transitive dependencies under untrusted parents with --all --trusted (isolated) [19.13ms]
(pass) should list nothing with --trusted when no dependencies are trusted [20.57ms]
597 |   expect(await exists(join(package_dir, lockfile))).toBeTrue();
598 |   urls.length = 0;
599 | 
600 |   const [stdout, stderr, exitCode] = await spawnAndCollect("pm", "ls");
601 |   expect(stderr).toBe("");
602 |   expect(normalizeBunSnapshot(stdout, package_dir)).toMatchInlineSnapshot(`
                                                          ^
error: expect(received).toMatchInlineSnapshot(expected)

  
  "<dir> node_modules (5)
  ├── bar@0.0.2
+ ├── bar@0.0.2
  ├── bar-alias@0.0.2
  ├── ws-once@workspace:packages/ws-once
+
... (truncated)
```

</details>

<details><summary>passes on PR (with fix)</summary>

```console
ASAN with fix: all passed
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/cli/install/bun-pm.test.ts
bun test v1.4.0 (6e906e4)

test/cli/install/bun-pm.test.ts:
(pass) should list top-level dependency [381.86ms]
(pass) should list all dependencies [333.18ms]
(pass) should list top-level aliased dependency [351.49ms]
(pass) should list aliased dependencies [347.67ms]
(pass) should list only trusted dependencies with --trusted [463.21ms]
(pass) should list only trusted dependencies with --all --trusted [348.99ms]
(pass) should list trusted transitive dependencies under untrusted parents with --all --trusted (isolated) [321.16ms]
(pass) should list nothing with --trusted when no dependencies are trusted [309.86ms]
(pass) should list a workspace the root also depends on once (bun.lock) [541.88ms]
(pass) should list a workspace the root also depends on once (bun.lockb) [314.14ms]
(pass) should list a trusted workspace the root also depends on once with --trusted [315.39ms]
(pass) should list a root optional peer that a dependency provides [328.74ms]
(pass) should remove all cache [465.82ms]
(pass) bun pm m
... (truncated)

release with fix: all passed
$ bun scripts/build.ts --profile=release
[configured] bun-profile → bun (stripped) in 683ms (unchanged)
ninja: Entering directory `/workspace/bun/build/release'
[1/5] gen generated_host_exports.rs
generated_host_exports.rs: 92 exports (host=3, lazy=10, generic=79, rust=0); 240 extern-C blocks audited
[1/5] cargo bun_bin → libbun_rust.a (--target x86_64-unknown-linux-gnu)

  nightly-2026-07-20-x86_64-unknown-linux-gnu unchanged - rustc 1.99.0-nightly (9f36de775 2026-07-19)

�[1m�[92m   Compiling�[0m bun_runtime v0.0.0 (/workspace/bun/src/runtime)
�[1m�[92m   Compiling�[0m bun_bin v0.0.0 (/workspace/bun/src/bun_bin)
�[1m�[92m    Finished�[0m `release` profile [optimized + debuginfo] target(s) in 4m 36s
[2/5] link bun-profile
[4/5] strip bun
[4/5] bun-profile --revision
1.4.0-canary.1+b263001b0
[build] done
bun test v1.4.0-canary.1 (b263001)

test/cli/install/bun-pm.test.ts:
(pass) should list top-level dependency [15.40ms]
(pass) should list all dependencies [11.19ms]
(pass) should list top-level aliased dependency [9.40ms]
(pass) should list aliased dependencies [9.71ms]
(pass) should list only trusted dependencies with --trusted [10.18ms]
(pass) should list on
... (truncated)
```

</details>

<details><summary>diff hotspot</summary>

```
src/runtime/cli/package_manager_command.rs |   8 +-
 test/cli/install/bun-pm.test.ts            | 150 ++++++++++++++++++++++++++++-
 2 files changed, 152 insertions(+), 6 deletions(-)
```

</details>

**gate history** · 1 passed · 0 rejected · iteration 0

<details><summary>evidence per changed file</summary>

```
file                                        reads  edits  tests
src/runtime/cli/package_manager_command.rs      5      4      0
test/cli/install/bun-pm.test.ts                10      9      0
```

</details>

**root cause** · written by the author bot

The root package's dependency list receives one entry per workspace
member plus one per declared dependency, so a workspace that the root
also declares as a dependency has two entries resolving to the same
package, and the default bun pm ls loop printed each of them. The fix
sorts the root's entries by name with a stable sort and then collapses
adjacent entries with the same name, so each package is printed once,
while the --trusted filtering in the same branch is unaffected.

<!-- robobun:evidence:end -->
@robobun
robobun force-pushed the farm/5a2216dc/peer-local-source branch from c172f8f to 4cb962d Compare August 22, 2026 07:29
@robobun

robobun commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator Author

@alii rebased once more onto today's main (4cb962d, one commit, mergeable). One conflict: lockfile.rs where #40014 and this PR both add helpers at the same spot; both kept.

#40014 (self-contained workspaces) adds a hoisting barrier the provider rule has to respect: a package that a self-contained workspace depends on gets its own copy inside that workspace's node_modules, where the root's folder is not visible, so Lockfile::dedupes_onto_root_dependency now also returns false when a self-contained workspace (transitively) depends on the declarer. The peer then resolves from the registry as on main and the registry copy lands inside the workspace, which keeps the workspace complete. One test in bun-install.test.ts; it fails with the barrier check removed (the peer binds to the folder and nothing is placed inside the workspace).

Everything else is unchanged. Locally, every install suite passes except four bun pm ls snapshot tests in bun-pm.test.ts that fail identically with main's src/ (the (5) vs (5 installed) header from #38952), so those are main's.

Comment thread src/install/lockfile.rs
Comment thread src/install/PackageManager/PackageManagerEnqueue.rs Outdated
@robobun
robobun force-pushed the farm/5a2216dc/peer-local-source branch from 4cb962d to 213a32d Compare August 22, 2026 08:42
Comment thread src/install/PackageManager/install_with_manager.rs Outdated
Comment thread src/install/PackageManager/install_with_manager.rs Outdated
@robobun
robobun force-pushed the farm/5a2216dc/peer-local-source branch from 213a32d to f09b280 Compare August 22, 2026 08:43
@robobun

robobun commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator Author

@alii one more change since this morning's rebase, now f09b280 (rebased onto the newest main, still one commit, mergeable). Review found two more cases where the root-folder arm could bind something the installed tree cannot reach, and one of them needed a structural fix rather than another clause:

  • The "nothing else installs a different package under the declarer's name" check reads the dependency graph during the peer pass, but a peer that has to come from the registry can pull new packages in afterwards, so the answer could be stale. Peers that only a root folder could still satisfy are now parked (find_peer_provider returns Deferred) instead of going to the registry, and wait_for_resolution decides them once the peer queue is empty and nothing is in flight (resolve_deferred_folder_peers). The predicate can only flip from true to false as edges arrive, so rejected peers go to the registry first and the rest are re-checked after the graph settles; a round that binds everything ends the loop. The own-dependency and tarball/git arms are unaffected (they do not depend on the rest of the graph).
  • bun install --filter=<workspace> can leave the root's dependencies out of node_modules entirely, so the root-folder arm is skipped for filtered installs.

Both have tests in bun-install.test.ts that fail on the previous head. All install suites pass locally apart from main's own bun pm ls / bun prune snapshot failures, which are reported separately. The description's Fix and Verification sections are updated.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed f09b280 and found no bugs. Given the new deferred-peer resolution loop introduced in this revision and the number of times the provider rule has needed narrowing, a human sign-off on the final shape is still warranted.

What was reviewed:

  • resolve_deferred_folder_peers loop termination — each round either binds all remaining peers or sends a strict subset to the registry; dedupes_onto_root_dependency is monotone (true→false only) as edges arrive, and the package set is finite.
  • dedupes_onto_root_dependency when the declarer is a self-contained workspace — workspace packages are Resolution::Workspace, so find_peer_provider never reaches the roots arm for their own peers; the predicate is only queried for the workspace's dependents.
  • peer_graph_complete is held true across the enqueue_dependency_with_main calls in the final round so bound peers actually resolve to the folder rather than re-deferring.
Extended reasoning...

Overview

This PR changes how bun install resolves peer dependencies that a file: package could satisfy, and how folder-resolved edges are placed in the lockfile tree. It touches PackageManager state (two new fields), the enqueue path (get_or_put_resolved_package, new find_peer_provider/bind_peer_provider), the resolution wait loop (wait_for_resolution now iterates until deferred folder peers settle), three new Lockfile helpers (declaring_package, resolution_of_dependency_named, dedupes_onto_root_dependency, is_dependency_of_self_contained_workspace), and Tree::process_subtree (folder edges now go through hoist_dependency). ~1,200 lines of tests across three files pin each clause of the provider rule.

Security risks

None identified. The change is confined to dependency-graph resolution and tree placement; it does not touch network trust, integrity verification, path validation, or archive extraction. The new code reads only lockfile buffers already populated by existing parsing.

Level of scrutiny

High. This is core package-manager resolution logic where a wrong binding silently installs the wrong package (or nothing) under a consumer. The provider rule has been narrowed through eight rounds of review on this PR, each finding a shape where the previous revision bound a folder the installed tree could not reach. The latest revision introduces a new structural mechanism (deferred_folder_peers + the resolve_deferred_folder_peers fixpoint loop in wait_for_resolution) that had not been through prior review rounds. That mechanism's correctness depends on dedupes_onto_root_dependency being monotone under graph growth and on the loop terminating, both of which I traced but which deserve human confirmation.

Other factors

alii previously approved and then the rule was narrowed three more times; robobun has already asked for a fresh approval on f09b280. All prior automated-review threads are resolved with tests that fail on the previous head. The one existing test expectation that changed (pnpm-lock-v9.test.ts) is explained in the description and looks correct — a peer satisfied by an ancestor workspace's copy no longer gets a redundant nested placement. Test coverage is thorough (each clause of the rule has a positive and a negative test), and every install suite is reported passing locally. Deferring rather than approving because the resolution-loop change is new enough that a maintainer familiar with the peer-pass ordering should confirm the fixpoint is sound.

Squash of the reviewed commits, rebased on main. Parts of the original
change landed separately while it was in review and are no longer in
this diff: the bun.lock parser change (#38851) and the tree dedupe of a
peer bound to a folder package (#40564). A self-contained workspace
(#40014) is a hoisting barrier, so a package in its dependency closure
does not take a root folder as its peer provider. Peers a root folder
could satisfy are decided only once the dependency graph is complete.
See the PR description for the full write-up.
@robobun robobun changed the title install: resolve peers provided by file: dependencies and stop writing duplicate package paths install: resolve peers provided by file: dependencies Aug 27, 2026
@robobun
robobun force-pushed the farm/5a2216dc/peer-local-source branch from f09b280 to 3835677 Compare August 27, 2026 09:21
@robobun

robobun commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator Author

@alii rebased onto today's main (3835677, one commit, mergeable). Two more pieces of this PR landed independently while it waited, so the diff got smaller:

What is left is the find_peer_provider rule with its helpers, the deferred decision for root-folder peers, and the tests. Title and description are updated to match (the PR no longer claims the duplicate-path symptom). With main's src/, 5 of the 7 bun-lock tests and 3 of the bun-install tests fail; the rest pin individual clauses. All install suites pass locally, including bun-pm and bun-prune now that main fixed their snapshots.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

@robobun

robobun commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator Author

Follow-up to my comment above: this PR no longer carries the Tree.rs folder dedupe after the rework, and the duplicate packages key for a file: folder in both dependencies and devDependencies still reproduces on 1.4.3, so that fix is now its own PR: #41833.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants