Skip to content

install: derive the bun.lockb meta hash in the writer - #44501

Draft
robobun wants to merge 4 commits into
mainfrom
robobun/a5ea2828/lockb-writer-derives-meta-hash
Draft

robobun wants to merge 4 commits into
mainfrom
robobun/a5ea2828/lockb-writer-derives-meta-hash

Conversation

@robobun

@robobun robobun commented Oct 3, 2026

Copy link
Copy Markdown
Collaborator

Problem

  • A bun.lockb written from pnpm-lock.yaml stores an all-zero meta hash. The next bun ci exits 1 with error: lockfile had changes, but lockfile is frozen.
  • The writer copies Lockfile.meta_hash (src/install/lockfile/bun.lockb.rs:186). The pnpm migrator never fills that field. The yarn and npm migrators fill it without lifecycle scripts, so a root postinstall fails the same check.

Fix

  • generate_meta_hash takes the script lines from the package columns. The writer hashes the packages it writes, unless the comparison before the save did.
  • Lockfile.meta_hash becomes loaded_meta_hash: Option<MetaHash>, which code only compares. Without a stored hash, the frozen check uses Lockfile::eql.
  • bun pm migrate and bun pm trust save through one entry. It prepares a migrated lockfile like the next install does.
  • Verified: 28 new tests in test/cli/install/bun-lockb.test.ts and test/cli/install/migration/, 20 fail on 1.4.3-canary.1. Notes list the other suites.

Background

  • The meta hash is SHA-512/256 over the sorted name@resolution lines and the lifecycle scripts of the root and workspaces. A frozen install compares the stored hash with a fresh one.
  • bun.lock stores no hash. Its frozen check is Lockfile::eql.
  • A review weighed one line in the pnpm migrator. The field then goes stale after bun add.

Downsides

  • bun pm migrate to bun.lockb does more work. Its save step takes 1.6M instructions and 52 allocations at 202 packages (base 0.15M, 18).
  • bun pm hash-print prints zeros in a project with only yarn.lock or package-lock.json.
  • Installs with bun.lockb pay nothing new: save_to_disk takes 871,346 instructions at 1,002 packages (base 871,273).
Notes

Measurements. Release builds of the merge base and of this branch, linux-x64. Counts come from gdb breakpoints and a ptrace single-step counter. strace, perf, valgrind and bloaty are not in the container, so syscall counts use gdb catchpoints and sizes use size.

  • generate_meta_hash calls per flow (base/PR): bun.lock no-op 0/0, fresh bun.lock 1/0, text --lockfile-only 1/0, bun.lock frozen 0/0, bun.lock add 0/0, bun.lockb no-op 1/1, bun.lockb fresh 1/1, bun.lockb frozen 1/1, bun.lockb add 1/1, bun.lockb remove 1/1, bun.lockb --lockfile-only 1/1, pnpm to bun.lockb install 0/1, pnpm to bun.lock install 0/0, npm to bun.lock install 2/2, npm to bun.lockb install 2/2, yarn to bun.lock install 2/2, bun pm ls on a yarn.lock project 1/0, bun pm ls on a pnpm project 0/0, bun pm migrate pnpm to bun.lockb 0/1, pnpm to bun.lock 0/0, npm to bun.lockb 1/1, yarn to bun.lockb 1/1, npm to bun.lock 1/0, bun install --yarn on bun.lock 0/1.
  • save_to_disk on bun.lockb during bun add at 1,002 packages: 871,346 instructions, 18 allocations (base 871,273, 18). At 202 packages: 141,008 and 18 (base 141,010 to 141,700 and 18).
  • no-op bun.lock install at 202 packages: 5,931,191 instructions inside install_with_manager (base 5,939,051). Fresh --lockfile-only install to bun.lock: 5,389,872 (base 5,879,068).
  • binding.gyp probes per install: 1 (base 1). Per bun pm migrate to bun.lockb: 1 (base 0). Per bun pm migrate to bun.lock: 0 (base 0).
  • bun pm migrate to bun.lockb: clean_with_logger calls 1 (base 0). Save step at 202 packages: pnpm 1,600,203 instructions and 52 allocations, npm 1,845,084 and 489 (base 146,858 and 18, 146,488 and 18). bun pm migrate to bun.lock: 1,893,448 instructions (base 1,893,390), same allocations.
  • known-answer files: 8 of 8 bun.lockb files written by the base build pass bun ci on the PR build. Bytes differing after a re-save by the PR build: 0. bun pm trust keeps the stored hash.
  • first-frozen sweep over every foreign-lockfile fixture in test/cli/install/migration, 275 rows: unchanged flips 0 of 97, dependency-deleted flips 0 of 81, root-hook rows 1 to 0: 52 (48 package-lock.json, 4 yarn.lock), 0 rows go 0 to 1.
  • bun.lockb producer rows failing the next frozen install: 0 of 123 (base 37 of 123). The matrix is pnpm, yarn, npm and no lockfile, times pm migrate, install, install --lockfile-only, add and remove, times seven project shapes (plain, root postinstall, root binding.gyp, root prepare plus binding.gyp, an unreferenced lock entry, a workspace postinstall, a workspace binding.gyp).
  • bun pm migrate to bun.lockb against bun install --lockfile-only on the same input, 14 cases: 14 store the same meta hash (base 3), 4 files are byte-identical (base 0). The others differ by one byte (configVersion, which install: lockfile parity for bun pm migrate, bun pm trust and yarn.lock entries without #sha1 #41908 changes) or two (a pnpm importer literal, which install: migrate pnpm importer rows in the shape package.json parsing produces #38791 changes).
  • size_of::(): 1,304 bytes (base 1,448). Release binary: text 80,663,855 bytes (base 80,679,983), data and bss unchanged (size).
  • hash-string blocks printed by --verbose per flow (base/PR): fresh bun.lock 1/1, bun.lock no-op 0/0, fresh text --lockfile-only 1/1, bun.lockb no-op 1/1, bun.lockb frozen 1/1, bun.lockb --lockfile-only 1/1, fresh bun.lockb 1/1, pnpm to bun.lockb 0/0, npm to bun.lock 1/1. The blocks are identical.

Behaviour that changes.

  • A bun.lockb with all-zero hash bytes and more than one package loads as "no stored hash". A frozen install compares it with Lockfile::eql, the check bun.lock uses, and a plain install writes the hash. Before, the frozen install failed until a plain install ran. Files from the pnpm migration of 1.2.23 to 1.4.2 are in this state.
  • The first frozen install from package-lock.json or yarn.lock passes when package.json has lifecycle scripts. The hash of the migrated lockfile is now taken from the same columns as the hash after the resolve. A dependency that package.json dropped still fails the check.
  • bun install --yarn prints the hash of the lockfile in the yarn.lock header when it loaded a bun.lock. Before, it printed zeros. bun bun.lockb does the same for a file without a stored hash.
  • bun pm hash and bun pm hash-string are unchanged. They hash packages only.
  • bun pm migrate and bun pm trust no longer abort a debug build when a migrated pnpm-lock.yaml has patchedDependencies and the output is bun.lockb. Two debug assertions required a patch file hash that only an install computes. They are gone. install: lockfile parity for bun pm migrate, bun pm trust and yarn.lock entries without #sha1 #41908 removes the same one in the writer.

Differences from the design review.

  • The review wanted the direct save to compute patch file hashes. This PR leaves them unset, as a release build of main does. Every install recomputes them before it uses them.
  • The review expected the first frozen install from package-lock.json to keep failing when a workspace has install scripts. It passes here, because the loaded side of the comparison is hashed from the lockfile after the differ has read the workspace scripts. A test covers it.
  • The review expected 2/1 hash computations for an npm to bun.lock install. The count stays 2/2: one for the migrated lockfile, one for the result.

Not covered. bun pm migrate to bun.lockb from a yarn v1 workspace project with a workspace lifecycle script. The yarn migrator creates no workspace rows, so the direct save has nothing to fill. This is from reading the code. I did not measure it.

Overlap with open PRs. #43912 adds Scripts::wants_default_node_gyp. This branch adds the same helper with the same text, so the two merge with one trivial conflict in load_root_lifecycle_scripts. #44294 conflicts in one line of package_manager_command.rs. #41908 touches save_to_disk, bun pm migrate and bun pm trust. #40057 adds a migrator that assigns meta_hash. That line goes away with the field.

Suites run with the debug build. bun-lockb, bun-pm, bun-lock, bun-workspaces, bun-install, bun-install-registry, bun-install-lifecycle-scripts, frozen-lockfile-pruned, frozen-lockfile-missing-workspace, migrate-bun-lockb-v2, lockfile-version-2, lockfile-only, overrides, config-version, bun-update-lockfile-sync and all of test/cli/install/migration/. Failures that also occur without this change: the bitbucket, gitlab and external tarball tests in bun-install.test.ts need the network. ensureTempNodeGypScript works and node -p should work in postinstall scripts clear PATH and need a binary named bun. manifest conditional requests > a changed etag returns 200 fails about one run in three on the debug build. Several files need more than the 5 s default per test on a debug build.

The stored hash was a field on Lockfile that each producer had to fill
before a save. The pnpm migrator never did, so a bun.lockb written from
pnpm-lock.yaml stored zeros and the next frozen install failed. The yarn
and npm migrators filled it before the lifecycle scripts were known, so
bun pm migrate to bun.lockb failed the same check when package.json had
a root lifecycle script.

generate_meta_hash now takes the script lines from the package columns,
and the writer computes the hash from the packages it writes unless the
comparison before the save already did. The hash a bun.lockb was loaded
with is kept only to compare. bun pm migrate and bun pm trust save a
migration as bun.lockb through one entry that first takes lifecycle
scripts from package.json and drops unreferenced packages, as the next
install does.
@github-actions github-actions Bot added the claude label Oct 3, 2026
@robobun

robobun commented Oct 3, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 10:21 PM PT - Oct 2nd, 2026

✅ @robobun, your commit 56ce66a5857e105b73bdadf4cb367510b70540d3 passed in Build #123135! 🎉


🧪   To try this PR locally:

bunx bun-pr 44501

That installs a local version of the PR into your bun-44501 executable, so you can run:

bun-44501 --bun

@robobun

robobun commented Oct 3, 2026

Copy link
Copy Markdown
Collaborator Author

Status: draft. The self-review of this diff is in progress.

How I reproduced it (1.4.3-canary.1 at 367d939, and a debug build of main):

  1. Make a project with two registry dependencies, a pnpm-lock.yaml (lockfileVersion 9.0) that locks them, and a bunfig.toml with [install] and saveTextLockfile = false.
  2. Run bun pm migrate. Bytes 46 to 78 of bun.lockb, the stored meta hash, are all zero.
  3. Run bun install --frozen-lockfile. It exits 1 with error: lockfile had changes, but lockfile is frozen.

The same steps from yarn.lock or package-lock.json pass, until package.json has a root lifecycle script. Then they fail in step 3 too.

PR: #44501

Comment thread src/install/PackageManager/PackageManagerDirectories.rs Outdated
Comment thread src/install/PackageManager/PackageManagerDirectories.rs Outdated
Comment thread src/install/PackageManager/PackageManagerDirectories.rs Outdated
Comment thread src/install/PackageManager/PackageManagerDirectories.rs Outdated
Comment thread src/install/PackageManager/PackageManagerLifecycle.rs Outdated
Comment thread src/install/PackageManager/install_with_manager.rs Outdated
Comment thread src/install/PackageManager/install_with_manager.rs Outdated
Comment thread src/install/PackageManager/install_with_manager.rs Outdated
Comment thread src/install/lockfile.rs Outdated
Comment thread src/install/lockfile.rs Outdated
Comment thread src/install/lockfile.rs Outdated
Comment thread src/install/lockfile.rs Outdated
Comment thread src/install/lockfile.rs Outdated
Comment thread src/install/lockfile.rs Outdated
Comment thread src/install/lockfile.rs Outdated
Comment thread src/install/lockfile.rs Outdated
Comment thread src/install/lockfile.rs Outdated
Comment thread src/install/lockfile.rs Outdated
Comment thread src/install/lockfile.rs Outdated
Comment thread src/install/lockfile/Package/Scripts.rs Outdated
Comment thread src/install/lockfile/bun.lockb.rs Outdated
Comment thread src/runtime/cli/pm_trusted_command.rs Outdated

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant