resolver: cache nothing for a failed directory read and fail the resolution that needed it

[robobun]

Problem

• A directory read that fails part-way is cached as the whole directory. import "dep" then loads dep/index.js, not the main of its package.json, with exit 0.
• The cause is Err(_) => break in the two read loops (src/resolver/resolver.rs:3392 and :4652). The two readers in src/resolver/lib.rs cache the error, and it never expires.

Fix

• RealFS::read_listing is the only reader and RealFS::commit_listing the only writer. A listing is stored only after the read reaches the end.
• A failed read is tried once more on a new handle. If it fails again, nothing is stored.
• The resolution then fails: Cannot read directory "<dir>": EIO while resolving "dep".
• Verified: test/js/bun/resolve/resolve.test.ts (11 new tests, 10 fail on 1.4.3), and the hot, watch, --filter and Bun.build suites.

Background

• The resolver lists a directory once. Later lookups use that listing and a DirInfo derived from it (package.json, tsconfig.json, node_modules).
• Considered: return the error from the two loops only. The other readers still cache a permanent error.
• Considered: answer by lstat when a listing fails. That is a second lookup path, and it reports nothing.

Downsides

• Each resolution pays one load and one untaken branch. The instruction counts are pending (Notes).
• An ancestor whose read keeps failing (not EACCES or EPERM) now fails the resolutions below it. Before, they used the partial listing.
• A failed listing refresh through a cached DirInfo is still "no such entry" for that lookup.

Notes

How to reproduce

There is no fault injection for getdents64 in the tree, so the tests build an LD_PRELOAD library. bun issues getdents64 through libc syscall(). For one directory, the first getdents64 of a handle returns the real records without one name, and each later getdents64 of that handle fails with EIO.

node_modules/dep/package.json   {"name":"dep","main":"real.js"}
node_modules/dep/real.js
node_modules/dep/index.js
entry.ts                        import which from "dep"; console.log(which)

Case	1.4.3	This branch
bun run entry.ts, the read of node_modules/dep fails each time	loads index.js, exit 0	error: Cannot read directory ".../node_modules/dep": EIO while resolving "dep", exit 1
the same, one failed read	index.js for the first import and for each later require, Bun.resolveSync, require.resolve	real.js for all four
"main": "lib/real.js", the read of dep/lib fails once	index.js	real.js
bun build entry.ts --target=browser, ./src/x with src/x.ts and src/x.js, the read of src fails each time	bundles x.js, exit 0	the error above, exit 1, no output file
the same with --target=bun	Cannot read directory ".../src": EIO and Could not resolve: "./src/x", exit 1	the same message as the browser target
Bun.build three times, the read of src fails during the second	second build: the two errors above. Third build: the same two errors	second build: the error. Third build: x.ts
bun run --filter '*' hello, one failed read of the workspace root	No workspace packages matched the filter "*", exit 1	runs the three scripts

What changed, by function

• RealFS::readdir is the one loop. RealFS::read_listing wraps it: it names the listing, reads, and on an error goes to the cold read_listing_again.
• read_listing_again opens the directory again by path and reads once more. The first handle is not read again, because its position is not known. When the caller owns the handle, the new handle replaces it. A non-void iterator (the bun test scanner) gets no second read, because it already saw entries.
• RealFS::read_failed is the rule for a read that failed twice. ENOENT and ENOTDIR (the directory went away while it was open) end like a failed open. Any other error stores nothing. A stale listing keeps its names and its generation. The handle is closed when the call opened it.
• EntriesMap::put and mark_not_found are private to the module. commit_listing, opaque_listing and mark_dir_not_found are the named writers that resolver.rs uses.
• dir_info_cached_miss: a read error of ENOENT or ENOTDIR ends like its failed open. EACCES or EPERM on an ancestor gives the empty listing that windows: run Bun inside an AppContainer (lowbox token) #33119 gives an ancestor that does not open. Any other error returns Err and leaves both cache keys unknown.
• The walk interns its path after the first directory is listed, and not after it opens. A walk that stops at a failed read leaves nothing in DirnameStore.
• RealFS.failed_listings keeps the entries that failed reads interned, for each directory, until a read of that directory succeeds. EntryStore, FilenameStore and DirnameStore only grow, so a read that fails again and again must not add to them.
• Resolver.dir_read_failure records the directory and the errno. dir_info_cached_miss, dir_info_for_resolution and load_as_file set it. resolve_and_auto_install tests it once after the resolution. If it is set, a cold function clears it and resolves once more, because a lookup outside a resolution can also leave a record. If the read fails again, the result is Failure with the message above, added with add_resolve_error as in Don't panic when auto-install can't read the top-level directory #31938.

Numbers

Counted on a debug build of this branch with gdb breakpoints and the shim, linux-x64.

	Value
Failed reads for one failed Bun.resolveSync("dep"), fault stays	8 (two passes, two lookups of the directory in each pass, each read tried twice)
Failed reads when one read fails	1, and the resolution succeeds
New Entry slots, DirnameStore strings and FilenameStore strings for each failed resolution after the first	0, 0, 0 (5 and 55 failed resolutions give the same totals: one directory, two directories in turn, a relative import, a failing ancestor)
Open fds after 402 failed resolutions, default, --hot, --watch	the same as before them

Release builds of the merge base and of this branch are in progress. The instruction count for a warm resolution, the size of the binary and the stack frames of the two walk functions follow before this leaves draft.

Suites run on the debug build

• test/js/bun/resolve/resolve.test.ts: 106 pass, 2 skip.
• test/js/bun/resolve/import-meta.test.js, resolve-error.test.ts, test/js/bun/util/filesystem_router.test.ts, test/cli/hot/hot.test.ts, test/cli/watch/watch.test.ts: all pass.
• test/cli/run/filter-workspace.test.ts: 88 pass, 1 fail ("run in parallel", which needs the output of two scripts to interleave in time; it passes alone).
• test/cli/run/env.test.ts: 106 pass, 1 fail (a compiled executable test, 5 s timeout).
• test/bundler/bun-build-api.test.ts: 67 pass, 3 fail (three bytecode tests, 5 s timeout each).
• test/cli/test/bun-test.test.ts: 102 pass, 2 fail (two deep-nesting print tests, 5 s timeout each).
• test/internal/source-lints/: 170 pass.
• test/js/bun/resolve/resolver-permission-denied-ancestor.test.ts skips as root here.
• cargo check -p bun_resolver for x86_64-pc-windows-msvc, aarch64-apple-darwin, x86_64-unknown-freebsd, aarch64-unknown-linux-musl.

The host was under heavy load, so the 5 s timeouts are not a signal. None of those tests reads a directory through the changed code in a way the others do not.

Not in this change

• The EINTR retry for getdents64 is in fs.rm: continue past vanished entries, retry transient errors, report the failing path #41480. Until it lands, one EINTR is healed by the second read.
• A failed open still caches errors such as EMFILE as "not found" (dir_info_cached_miss, read_directory_error).
• bun run <file> prints Module not found for an entry point whose directory cannot be read, without the errno. bun run --filter prints error: Unexpected for EIO.
• The bun test scanner only debug-logs a directory it cannot read (bun test: report directories the scanner cannot read, scan each directory once #41465).
• A DirInfo built while the stale listing of its parent cannot be refreshed has no real path for a symlinked directory.
• Other loops over the same directory iterator that stop at an error: bun pm pack and bun publish (pack, publish: exit 1 when a read of a package directory fails #44400), bun pm licenses, the system certificate loader, install prune.

Related open PRs

• resolver: keep cached DirEntry when a stale-generation refresh fails #36874 (keep the cached listing when a refresh fails) and resolver: close the store_fd directory handle when readdir fails #36878 (close the directory handle when a read fails): this change does both in read_failed and in the callers of read_listing.
• fs: validate POSIX directory records in one parser, report a malformed record as EIO #44195 makes the iterator report a malformed record as EIO. That error goes through the same rule.
• resolver: rewind cached directory fds before re-iterating them #36245 (rewind cached directory fds) is not needed for the second read here, because it opens by path.

Two questions

• The one load and one branch for each resolution. If that is not acceptable, the cache half stands alone: drop dir_read_failure and the check, and a read that keeps failing is then only "not found" for that resolution.
• The ancestor rule. The other choice is to treat every read error on an ancestor as the empty listing that EACCES gets.

[robobun]

Status

Reproduced on 1.4.3 (release build) and on a debug build of main, linux-x64. An LD_PRELOAD library makes one getdents64 of node_modules/dep fail with EIO after the first read of the directory returned its records without package.json. bun run entry.ts then loads node_modules/dep/index.js and not the main of package.json, with exit 0. One failed read gives the same wrong file to each later require, Bun.resolveSync and require.resolve in the process.

The fix and the tests are in this PR (#44540). It is a draft until the release-build numbers are in the body.