Skip to content

install: resolve deferred * peers after their siblings, not on arrival order - #37713

Open
robobun wants to merge 4 commits into
mainfrom
farm/74487e95/peer-star-deterministic
Open

robobun wants to merge 4 commits into
mainfrom
farm/74487e95/peer-star-deterministic

Conversation

@robobun

@robobun robobun commented Aug 12, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • bun-install-registry.test.ts hoisting > peers > it should hoist 1.0.1 when peer * goes red on unrelated PRs, mostly on Windows (latest: build 92591, 4/4 attempts): Expected to contain: "1.0.1", received a-dep 1.0.9.
  • Cause: which a-dep a peer * edge binds to depends on registry response order. If the a-dep manifest is already in memory when the peer is seen, it binds at once to the highest a-dep in the lockfile at that instant; otherwise it waits for the peer pass and gets the highest overall.
  • Every other peer range is already protected from this; * was exempted because the peer * test was believed to need it. It does not: the test normally gets 1.0.10, which toContain("1.0.1") also matches.
  • The bun.lock loader had the same * exemption, so a reloaded * peer can bind to a different version than the resolver chose. When it does, the isolated linker adds a second store entry for the dependent on every warm install (2 packages installed instead of (no changes)). This part is already reachable on main.

Fix

  • Drop the * exemption in the resolver: a deferred peer binds early only on an exact match and otherwise waits for the peer pass, like every other range. The result depends only on the full set of siblings, never on arrival order (1.0.10 here).
  • Drop the matching exemption in the bun.lock loader, so a reload runs the same version scan as the resolver and the two agree.
  • An older bun.lock with a * peer bound lower rebinds once to the highest version, as non-* peers already do; --frozen-lockfile still passes on it.
  • Verification: a new test forces the CI arrival order through a proxy in front of the registry; it fails without the fix on every run (20/20 locally, same 1.0.9 as CI) and passes 30/30 with it. A second new test fails on main for the reload case. Stopgap until Deterministic dependency resolution: an ordered walk over a live tree #36476; test(install): mark "peer *" hoisting case todoIf(isFlaky) #33972 can be closed in favour of this.

Background

  • A peer dependency asks the dependent's tree to supply some version of a package. bun binds a peer edge either at once, if a matching package is already in the lockfile when the edge is seen, or in a later install_peer pass after every regular dependency has been appended.
  • Manifests download concurrently and packages are appended to the lockfile as each response is processed, so what is "already in the lockfile" at a given moment varies between runs and machines.
  • Hoisting: root's node_modules/a-dep is whichever a-dep the first sorted root dependency resolved to. Here peer-a-dep-star sorts first, so its binding is what the test reads.
  • bun.lock records the tree, not peer bindings; on load each peer edge is re-derived, by walking the printed tree or by the resolver's version scan. The isolated linker keys store entries by the peer versions a package resolved to, so both derivations must agree or warm installs churn.

no test proof · iteration 0 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/cli/install/bun-install-registry.test.ts

Original description

Fixes the bun-install-registry.test.ts flake that has been going red on unrelated PRs for weeks, mostly on the Windows lanes (latest: build 92591, 4/4 attempts):

hoisting > peers > it should hoist 1.0.1 when peer *
Expected to contain: "1.0.1"
Received: "{\n    \"name\": \"a-dep\",\n    \"version\": \"1.0.9\"\n}"

Cause

Root depends on uses-a-dep-1..10 (each pins a-dep@1.0.N) and peer-a-dep-star (peer a-dep@*). peer-a-dep-star sorts first among the root dependencies, so whatever its peer resolves to is what gets hoisted to node_modules/a-dep.

How the peer resolves depends on what the registry answered first. If the a-dep manifest is not in memory yet when peer-a-dep-star is processed, the peer is deferred and resolved in the peer pass after every sibling has been appended, where the descending package index gives 1.0.10. If the manifest is already in memory (slow machine: the a-dep manifest, requested after the first uses-a-dep-* response, lands before the remaining initial responses), get_or_put_resolved_package_with_find_result binds the peer immediately to the highest a-dep that exists in the lockfile at that instant. On the failing machines that was 1.0.9, because uses-a-dep-10 had not been processed yet.

suppress_peer_satisfies already prevents exactly this for every other range (>=, ^, ~, exact), but * was explicitly exempted, with a comment saying the peer * test depended on it. It does not: the test normally gets 1.0.10 and only passed because toContain("1.0.1") matches "1.0.10".

The text lockfile loader (is_deferred_peer in bun.lock.rs) carried the same * exemption: on reload, a * peer edge was rebound by the printed-tree path walk (nearest hoisted a-dep) instead of the version scan the resolver uses. Whenever the two differ, the isolated linker keys a second store entry for the dependent on every warm install. That part was already reachable on main, since the resolver usually deferred * peers too: with uses-a-dep-1, uses-a-dep-5 and a workspace depending on peer-a-dep-star, the second bun install --linker isolated reports 2 packages installed and adds a second peer-a-dep-star@1.0.0+<hash> entry to node_modules/.bun.

Fix

  • PackageManagerEnqueue.rs: drop the * exemption, so a deferred peer only binds early on an exact match of the manifest's best version and otherwise waits for the peer pass like every other range. The result no longer depends on arrival order (1.0.10 here).
  • bun.lock.rs: drop the matching exemption in is_deferred_peer, so loading bun.lock binds * peers with the same version scan and agrees with the resolver.

Loading a bun.lock written before this change whose * peer had been bound to a lower version rebinds it to the highest one in the tree, the same one-time adjustment non-* peers already get; --frozen-lockfile still passes on such a lockfile (checked with a lockfile produced by the forced arrival order below).

This is a stopgap. #36476 reworks resolution so arrival order cannot matter anywhere; #33972 (marks the case todo) can be closed in favour of this.

Tests

  • hoisting tests compare the exact a-dep version instead of toContain, and peer * expects 1.0.10.
  • New peer * hoists the same version no matter which manifest the registry answers first: a small proxy in front of verdaccio holds peer-a-dep-star's manifest until an a-dep tarball has been requested (so the a-dep manifest is loaded and 1.0.1..1.0.9 exist) and holds uses-a-dep-10's manifest until the peer-a-dep-star tarball has been requested. Each release is triggered by a request bun only makes after the previous step, so there are no timers. Without the fix it fails with 1.0.9 on every run (20/20 locally, the same value CI reports); with it, 30/30 pass.
  • New peer * binds to the same version when bun.lock is loaded as when it was resolved: the isolated-linker scenario above with --save-text-lockfile. On main the second install prints 2 packages installed instead of (no changes); with the fix the store entry set is unchanged.
  • The macOS todoIf on the peer ^1.0.2 case was for the same class of flake and is removed; that range has been covered by suppress_peer_satisfies since the rewrite.
  • bun-install-registry.test.ts, bun-install.test.ts, bun-add.test.ts, bun-lock.test.ts, isolated-install.test.ts, bun-workspaces.test.ts, test-dev-peer-dependency-priority.test.ts, the pnpm/yarn migration suites and regression/issue/36577.test.ts pass locally with the debug build (the only failures in bun-install.test.ts are the git/tarball tests that need network access and fail identically without this change).

…val order

A peer dependency whose manifest was already in memory when its parent
was processed got bound right away to whichever version of the package
happened to be in the lockfile at that instant. For every range except
`*` this was already suppressed so the peer is resolved in the deferred
peer pass, once all siblings have been appended. Extend that to `*`,
which is the range most exposed to arrival order since it is satisfied
by every sibling.

This is what made "hoisting > peers > it should hoist 1.0.1 when peer *"
flaky: on slow machines the a-dep manifest was processed before
peer-a-dep-star's, so the peer bound to a-dep@1.0.9 and that is what got
hoisted. The test only passed elsewhere because the peer normally ends
up on 1.0.10 and toContain("1.0.1") matches "1.0.10". Compare the exact
version now, expect 1.0.10 for `*`, and add a test that forces the slow
arrival order through a small proxy in front of the registry. The macOS
todo on the `^1.0.2` case covered the same class of flake and is removed.
@coderabbitai

coderabbitai Bot commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Changes

Peer dependency resolution

Layer / File(s) Summary
Exact peer binding
src/install/PackageManager/PackageManagerEnqueue.rs
Deferred peers bind immediately only when the manifest version matches exactly. Other peers use the later deterministic resolution phase.
Lockfile peer deferral
src/install/lockfile/bun.lock.rs
Non-optional wildcard peers now use version-based resolution. Optional peers retain hoisted-tree handling.
Resolution test coverage
test/cli/install/bun-install-registry.test.ts
Tests compare exact versions and cover wildcard selection, lockfile reloads, reinstall behavior, registry response ordering, and peer-store reuse.

Suggested reviewers: jarred-sumner

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: deferred wildcard peer dependencies resolve after sibling processing instead of arrival order.
Description check ✅ Passed The description clearly explains the problem, fix, affected components, tests, and verification results, despite using custom headings.

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Aug 12, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 8:20 PM PT - Aug 11th, 2026

✅ @robobun, your commit 61dfaf9c8e7826243e7c4d1f6a65666b427eddae passed in Build #92770! 🎉


🧪   To try this PR locally:

bunx bun-pr 37713

That installs a local version of the PR into your bun-37713 executable, so you can run:

bun-37713 --bun

@robobun

robobun commented Aug 12, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: fix pushed (resolver + bun.lock loader), waiting on CI.

Reproduced both halves deterministically in test/cli/install/bun-install-registry.test.ts:

  • peer * hoists the same version no matter which manifest the registry answers first forces the arrival order the slow lanes hit; on the canary it fails with Received: "1.0.9" (the value build 92591 reported) on 20/20 runs, 30/30 pass with this change.
  • peer * binds to the same version when bun.lock is loaded as when it was resolved fails on main with 2 packages installed on the warm isolated install (second store entry for peer-a-dep-star), passes with this change.

Supersedes #33972, which only marks the case todo. #36476 is the long term fix for arrival-order dependence in general.

Comment thread src/install/PackageManager/PackageManagerEnqueue.rs
The text lockfile loader kept the same `*` exemption as the resolver, so
a `*` peer edge was rebound to whatever the printed tree hoisted nearest
to it, while the resolver binds it to the highest version in the tree.
Whenever those differ the isolated linker keys a second store entry for
the dependent on every warm install. Route `*` peers through the same
version scan as every other peer range, matching the resolver.
Comment thread src/install/PackageManager/PackageManagerEnqueue.rs Outdated
Comment thread src/install/lockfile/bun.lock.rs Outdated
Comment thread src/install/PackageManager/PackageManagerEnqueue.rs
Comment thread src/install/lockfile/bun.lock.rs

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/install/PackageManager/PackageManagerEnqueue.rs (1)

1986-2007: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Reuse suppress_peer_satisfies in the fallthrough branch.

Line 1986 and line 2005 compute the same predicate behavior.is_peer() && !install_peer. The two sites must stay in sync: line 1986 disables the range match, and line 2005 defers the dependency. If one changes later, deferred peers bind by exact resolution but never defer, or defer without the exact-match attempt.

♻️ Proposed refactor
-    } else if behavior.is_peer() && !install_peer {
+    } else if suppress_peer_satisfies {
         return Ok(None);
     }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/install/PackageManager/PackageManagerEnqueue.rs` around lines 1986 -
2007, Reuse the existing suppress_peer_satisfies variable in the fallthrough
condition after get_package_id instead of recomputing behavior.is_peer() &&
!install_peer, keeping both range-matching and dependency-deferral paths
governed by the same predicate.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/cli/install/bun-install-registry.test.ts`:
- Around line 3735-3738: Remove the conditional console.log block around the
installed-output assertion, leaving the expect(out).not.toContain("package
installed") assertion unchanged.

---

Outside diff comments:
In `@src/install/PackageManager/PackageManagerEnqueue.rs`:
- Around line 1986-2007: Reuse the existing suppress_peer_satisfies variable in
the fallthrough condition after get_package_id instead of recomputing
behavior.is_peer() && !install_peer, keeping both range-matching and
dependency-deferral paths governed by the same predicate.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 1ddeee79-07c9-4061-ac0e-c428c8e3d5fd

📥 Commits

Reviewing files that changed from the base of the PR and between 0825a8b and 69fcc80.

📒 Files selected for processing (3)
  • src/install/PackageManager/PackageManagerEnqueue.rs
  • src/install/lockfile/bun.lock.rs
  • test/cli/install/bun-install-registry.test.ts

Comment thread test/cli/install/bun-install-registry.test.ts Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the mirrored is_deferred_peer exemption in 69fcc80 — the loader and resolver now agree, and the new bun.lock round-trip test covers exactly the store-entry re-keying I flagged. I re-reviewed and found no further issues. Since this changes the user-visible resolution of * peers (now deterministically the highest sibling in the tree rather than arrival-order-dependent) and the PR frames itself as a stopgap ahead of #36476, a maintainer sign-off on the intended semantics would still be worthwhile.

What was reviewed:

  • suppress_peer_satisfies now covers * — deferred peers fall through to the is_peer && !install_peer return and bind in phase 2's descending-index scan; the exact-match reuse path is preserved.
  • All three is_deferred_peer call sites in bun.lock.rs now route * peers through resolve_peer_dep_version_based; optional-peer exemption kept, matching process_subtree's hoisted-sibling binding.
  • Test tightening: toContain("1.0.1") → exact .toBe() via hoistedADepVersion(); the removed isMacOS import has no remaining references; the proxy test releases each held response on a request bun only makes after the prior step (no timers), and the stray console.log is gone.
Extended reasoning...

Overview

Two-line logic change in each of two files, plus test hardening. PackageManagerEnqueue.rs drops the is_star() exemption from suppress_peer_satisfies, so a still-deferred * peer can only bind early on an exact eql(find_result) match and otherwise falls through to return Ok(None) at line 1997-1998, to be resolved in the install_peer pass by get_or_put_resolved_package's descending-index scan. bun.lock.rs drops the same exemption from is_deferred_peer, so the text-lockfile loader routes * peers through resolve_peer_dep_version_based (the loader-side replica of that scan) instead of the printed-tree path walk. The test file tightens the existing hoisting assertions from toContain to exact-version toBe, changes the peer * expectation from 1.0.1 to 1.0.10, removes the macOS todoIf on peer ^1.0.2, and adds two new tests: a proxy that forces the flaky arrival order, and a bun.lock/isolated-linker round-trip.

Security risks

None. No untrusted-input parsing, no allocation sizing, no auth/crypto. The change narrows an early-binding condition; the fall-through path already existed and handles every other peer range.

Level of scrutiny

High — this is bun install peer resolution, a critical path every user hits, and it changes what * peers resolve to. The old behavior was non-deterministic (the whole point of the PR: it flaked on Windows CI depending on which manifest arrived first), so any deterministic outcome is strictly better, but "highest sibling" vs. e.g. "lowest sibling" or "nearest hoisted" is a semantic choice a maintainer should confirm, especially since the PR itself calls this a stopgap ahead of #36476's rework.

Other factors

My previous review flagged that the resolver change would desync from is_deferred_peer in the lockfile loader; the author fixed that in 69fcc80 and added a dedicated test (peer * binds to the same version when bun.lock is loaded as when it was resolved) that fails on main with the isolated linker creating a duplicate store entry. The comment-cop and CodeRabbit nits (long comments, stray console.log) were addressed in 35fdb95 and 61dfaf9. All review threads are resolved. The proxy test is well-constructed — it uses Promise.withResolvers gates keyed on request paths bun only issues after the prior step completes, so it has no timers and no sleep. I checked that the removed isMacOS import has no other references in the file, and that all three is_deferred_peer call sites in bun.lock.rs benefit from the change. I did not find any other is_star() special-casing of peers in the install crate that would need the same treatment.

@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

This same * peer binding drift was surfaced again by an install fuzz scenario (isolated linker, peer prt-a@* not provided by the root, prt-a@1.0.0 hoisted to the root of the saved tree and prt-a@2.0.0 nested: the fresh install wires the dependent to 2.0.0, every install driven by the byte-identical bun.lock rebinds it to 1.0.0 and adds a second store entry). I reproduced it on current main and landed on the identical two-line fix before finding this PR, so I am not opening a second one.

Two things that may save whoever rebases this some time:

Suites run locally with the rebased change (debug build): isolated-install, bun-install-registry, bun-lock, bun-lockb, bun-workspaces, catalogs, bun-dedupe, bun-prune, bun-pm-licenses, bun-pm-why, bun-update-transitive, frozen-lockfile-pruned and the pnpm migration suites, all green.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants