Repository navigation
Conversation
When no install cache directory can be used, the package manager keeps its cache below the working directory: in node_modules/.bun-cache when nothing names a directory, in node_modules/.cache when the cache is off or the named directory cannot be made. `bun install` installs into node_modules, so that is where its cache belongs. The runtime's auto-install is on only while no node_modules directory exists. Its own cache made one. The first run loaded the package, and every later run failed with "Cannot find package". So did a later import in the same process, after any failed resolve. The runtime now keeps that cache in .bun-cache in the working directory. `bun install` is unchanged. The commands that walk a project and skip node_modules skip .bun-cache too: bun fmt, bun lint, bun run --filter, bun pm pack, bun pm diff, bun create and the framework router.
|
Status: draft. Reproduction on main: a directory with With this branch, three runs load the package. The registry gets 2, 0 and 0 requests, and the directory holds |
A process that starts without USERPROFILE on Windows gets the variable of its parent, so "nothing names a cache directory" cannot be set up there. That case is skipped on Windows. The other cases use a home directory that cannot be made.
|
This PR needs one decision before I change it further. @Jarred-Sumner, the question is yours. Is a visible A review of the diff kept the bug and the layer, and it questioned the shape. The fix itself is +19/-4 in What I checked on a debug build of this branch:
If yes, I do this here:
If no, I rework this PR to the resolver rule: a The PR stays a draft until then. The body lists what is open. |
Problem
HOME, a directory that cannot be made,[install.cache] disable = true), a script that imports a package works once. Later runs fail witherror: Cannot find package 'foo' from '<script>'.node_modules/.bun-cacheornode_modules/.cache(ensure_cache_directory,src/install/PackageManager/PackageManagerDirectories.rs). Anode_modulesdirectory turns auto-install off.Fix
.bun-cachein the working directory.bun installis unchanged.node_modulesskip.bun-cachetoo:fmt,lint,run --filter,pm pack,pm diff,create, the framework router..bun-cacheacceptable? See the comment below.test/cli/run/run-autoinstall-cache-dir.test.tsandtest/bake/framework-router.test.ts. All fail on main.Background
node_modulesdirectory at or above a script, bun installs imported packages at run time into its cache.node_moduleswith only bun's cache: bun still createsnode_modules. Considered the temp directory: cached packages resolve from<tmp>/node_modules, which other users can write.Downsides
.bun-cacheis visible beside the script. Prettier, ESLint andgit statussee it. They skippednode_modules.bun pm cache, which still makesnode_modules/.bun-cache. Anode_modules/.bun-cachefrom an older bun also keeps auto-install off..textgrows by at most 2,048 bytes (see Notes).bun pm packtests one more name per entry: no time change above noise.Notes
Who hits it. Containers and CI where
HOMEis unset,/or read-only, and anyone with[install] cache = falseor[install.cache] disable = true. The same happens whenBUN_INSTALL_CACHE_DIRnames a file, a name of 300 letters, or a path below/proc.In one process. After the first import makes
node_modules, any failed resolve makes the resolver read the directory again. Later imports then fail withCannot find module.Reproduction. A directory with
index.js(import "no-deps") and abunfig.tomlthat names a local registry. Runenv -u HOME -u BUN_INSTALL -u BUN_INSTALL_CACHE_DIR -u XDG_CACHE_HOME bun index.jstwo times. On main the first run loads the package and makesnode_modules/.bun-cache. The second run fails and sends 0 requests. With this change, three runs load the package, the registry gets 2, 0 and 0 requests, and the directory holds.bun-cache.Measurements. Release builds of main
a9c57bdd89and of this change on top of it, linux x64. The measured build also made the failure to open the cache directory a resolve error. #45307 has since done that on main and this diff no longer has it, so the size is an upper bound. The counts are libc calls logged by anLD_PRELOADshim (mkdir,mkdirat,open*,readlink*,renameat,symlinkat).mkdirat(<cwd>/node_modules)during a runtime import: 1 on main, 0 here, in each of 3 cases (noHOME,HOMEbelow a file,cache = false).mkdiratcalls to make the fallback cache: 3 on main, 1 here. Logged calls of the first run: 16 on main, 14 here.HOMEset): the logs of run 1 (18 calls) and run 2 (6 calls) are identical on both builds.bun installon its own fallbacks (noHOME, andcache = false): 16 calls each, identical logs.node_modulesstill holds.bun-cacheor.cache.require.resolve("no-deps")log 7 calls on both builds. The changed function,ensure_cache_directory, is#[cold]and runs once per process. Instructions were not counted:perfandvalgrindare not installed here.bun pm pack --dry-run, 1,001 files, 40 interleaved runs: median 22.3 ms here, 19.6 ms and 24.0 ms for two series of main.size -A):.text80,756,901 to 80,758,949 bytes..rodatais unchanged.On top of #44968.
fetch_cache_directory_pathstill returnsnode_modules/.bun-cachewithis_cwd_fallbackset. The runtime replaces the path when that flag is set.bunxkeeps its own rule for the flag.After #45307. That pull request made the process exits of the cache and temp directories (
bun is unable to write files) errors of the import. This change does not touch them. When.bun-cachecannot be made, the import fails with that error.Not in this change.
bun pm cachewith no named cache directory still makesnode_modules/.bun-cache, and auto-install is then off again (see below).bun pm cache rmdoes not clear.bun-cache.node_modulesthat holds only the cache of another tool (prettier --cachewritesnode_modules/.cache/prettier) turns auto-install off. That is the documented rule..bun-cacheare watched by--watchand--hot, like files from the global cache. Files innode_modulesare not.Open after self-review. Checked on a debug build of this branch unless it says otherwise.
src/runtime/bake/FrameworkRouter.rsandsrc/runtime/bake/bake_body.rs, and this diff edits both.bun pm cacheprints and makes<cwd>/node_modules/.bun-cache, and the third run fails witherror: Cannot find package 'no-deps'. The review measured the same forbun outdated,bun pm fetch,bun install --lockfile-onlyand four more commands. I ranbun pm cacheonly.BUN_INSTALL_CACHE_DIR=node_modules/.bun-cacheis replaced by<cwd>/.bun-cache: the runtime recognises the fallback by its spelling (is_cwd_fallback)../node_modules/.bun-cacheis used as named, and the second run fails. So the new docs sentence aboutBUN_INSTALL_CACHE_DIRis wrong for the first value.bun -iwith a writablenode_modulesin a directory that the user cannot write loaded the package before and fails now withEACCES. The second arm no longer uses anode_modulesthat exists..bun-cacheare not undernode_modules, so they are watched, and they can invoke macros. The global cache has the same property.Tests. The 21 cases of
run-autoinstall-cache-dir.test.tsinclude the ones forfmt,lint,run --filter,pm pack,pm diffandcreate. They are in their own file, and not concurrent: a debug build under load runs some cases ofrun-autoinstall.test.ts,format.test.tsandlint.test.tspast the default timeout of 5 s. On the two release builds, 0 of 22 and 22 of 22 new cases pass. On a debug build of this branch (mainf0e4f6b3b5), 21 of 21 and 36 of 36 pass. That was before #45489 moved the router files, so the router case has not run against main's files.Other suites, on the same two release builds:
run-autoinstall.test.ts,autoinstall-cached-manifest.test.ts,resolve-autoinstall-*.test.ts,run-autoinstall-abs-path.test.ts,bun-pm.test.ts,bun-pack.test.ts,bun-pm-diff.test.ts,bun-create.test.ts,filter-workspace.test.ts,init.test.tsandlint/defaults.test.tspass on both.format.test.ts,lint.test.tsandlint/config-files.test.tshave cases that time out on this machine on both builds. The cases that failed only with this change pass when they run alone.bun run rust:check-all x86_64-pc-windows-msvc aarch64-apple-darwinpassed for the measured build. CI ran the new file on Windows and macOS x64. On Windows one case is skipped: a process that starts withoutUSERPROFILEgets the variable of its parent, so the cause "noHOME" is POSIX only.