Read --env-file FIFOs until EOF instead of trusting stat.size#30521
Read --env-file FIFOs until EOF instead of trusting stat.size#30521robobun wants to merge 7 commits into
Conversation
FIFOs (and other non-regular files) always report stat.size == 0, so loadEnvFileDynamic / loadEnvFile would short-circuit and record the file as 'loaded' with empty content. This broke 1Password's local-env-file integration, which exposes secrets through a FIFO. Fix: factor out readEnvFileContents. When stat.kind != .file, fall back to a read-until-EOF loop that grows an ArrayList in 4 KiB chunks instead of preallocating from stat.size. Regular files keep the existing fast path that preallocates from stat.size. Fixes #30520
|
Updated 7:50 PM PT - May 11th, 2026
❌ @robobun, your commit bdae676 has 3 failures in
🧪 To try this PR locally: bunx bun-pr 30521That installs a local version of the PR into your bun-30521 --bun |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughCentralizes env-file reading into readEnvFileContents (reads sized or until-EOF depending on file type), refactors loadEnvFile and loadEnvFileDynamic to use it (handling empty-file nulls), and adds POSIX FIFO tests to validate FIFO-backed env-file reading. ChangesFIFO-aware env-file reading
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Warning Review ran into problems🔥 ProblemsTimed out fetching pipeline failures after 30000ms Comment |
Ban-words CI caught the helper's `std.fs.File` parameter pushing the count from 90 to 91. Swap the signature to `bun.FD` and route all reads/stats through bun.sys: - POSIX: bun.sys.fstat + bun.isRegularFile(stat.mode) for the regular-vs-non-regular branch, bun.sys.read / bun.sys.readAll for I/O. - Windows: bun.sys.getFileSize (GetFileSizeEx directly, avoiding the libuv ownership transfer that bun.sys.fstat does) + bun.sys.readAll. Caller still owns the std.fs.File handle and closes it. Callers convert via bun.FD.fromStdFile(file) at the call site. Error handling consolidates: OOM propagates, any other I/O error logs and marks the file as 'loaded' empty so it isn't retried — matches the original intent of the narrower error-set catch.
Previous build hit a known WebKit AtomStringImpl race in fetch-http2-client.test.ts (open PR #29453 addresses it) — unrelated to this diff, which only touches src/dotenv/env_loader.zig and test/cli/run/env.test.ts.
There was a problem hiding this comment.
Additional findings (outside current diff — PR may have been updated during review):
-
🟡
src/dotenv/env_loader.zig:893-901— Minor: the refactor dropped the caller-scopeerrdefer this.allocator.free(buf)that previously coveredtry Parser.parse(...)(andtry this.custom_files_loaded.put(...)inloadEnvFileDynamic). TheerrdeferinsidereadEnvFileContentsgoes out of scope once it returns, so if those later calls returnerror.OutOfMemory,read.bufis leaked. It's effectively unreachable in practice (OOM during startup env loading), but addingerrdefer this.allocator.free(read.buf);right after theorelseunwrap in both call sites would restore parity with the old code.Extended reasoning...
What changed
Before this PR, both
loadEnvFileandloadEnvFileDynamicallocatedbufdirectly at caller scope and immediately followed it witherrdefer this.allocator.free(buf);. Thaterrdeferremained active for the rest of the function body — in particular acrosstry Parser.parse(...)(which returnsOOM!void) and, inloadEnvFileDynamic, acrosstry this.custom_files_loaded.put(...)(also OOM-fallible). If either of those returned an error, the buffer was freed on the way out.After the refactor, allocation moved into
readEnvFileContents(). That helper has its ownerrdefer this.allocator.free(buf);(anderrdefer list.deinit();for the FIFO path), but Zig'serrdeferis scoped to the enclosing block — oncereadEnvFileContentsreturns successfully, thoseerrdefers are gone. Back at the call sites, afterconst read = read_result orelse { ... };there is noerrdeferprotectingread.buf. The subsequenttry Parser.parse(...)andtry this.custom_files_loaded.put(...)can still return an error, and if they do,read.bufis now leaked where it previously was freed.Step-by-step
loadEnvFileDynamicis called with--env-file=/tmp/fifo.readEnvFileContentsreads ~16 KiB from the FIFO into anArrayList, callstoOwnedSlice(), and returns{ .buf = owned, .amount_read = 16384 }. Its internalerrdefer list.deinit()is discharged on successful return.- Caller unwraps:
const read = read_result orelse { ... };—read.bufnow owns 16 KiB with no cleanup registered. Parser.parse(source, this.allocator, ...)runs and hitserror.OutOfMemoryinsideallocator.dupewhile inserting a value into the map.- The
trypropagates the error out ofloadEnvFileDynamic. Noerrdefercoversread.buf, so the 16 KiB allocation is leaked. The pre-PR code freed it via the caller-scopeerrdefer.
The same applies to
loadEnvFileat thetry Parser.parse(...)site, and toloadEnvFileDynamicat the trailingtry this.custom_files_loaded.put(file_path, source.*).Why it doesn't matter much
The only error these calls can return is
error.OutOfMemory. Bun's allocator behavior on OOM is effectively fatal — the process is not going to recover from OOM during startup env-file loading and continue running with a meaningfully observable leak. On the success path the buffer is intentionally never freed anyway (it backs thelogger.Sourcestored on theLoaderfor the program's lifetime), so this is purely about the OOM error path. This is a strict behavioral regression from the pre-PR code, but the practical impact rounds to zero.Fix
Add
errdefer this.allocator.free(read.buf);immediately after theorelseunwrap in bothloadEnvFile(afterconst read = read_result orelse { ... };around line 897) andloadEnvFileDynamic(around line 947). That restores the original cleanup contract with a one-line change at each site.
The pre-PR code allocated `buf` at caller scope and set `errdefer this.allocator.free(buf)` that stayed active across the following `try Parser.parse(...)` (and `try custom_files_loaded.put` in loadEnvFileDynamic). Moving the alloc into readEnvFileContents discharged that errdefer on successful return, so an OOM from the Parser/map insert would leak `read.buf`. Add `errdefer this.allocator.free(read.buf)` after the `orelse` unwrap in both call sites to restore the original cleanup contract. Reported by claude[bot].
open() on POSIX succeeds on directories opened read-only. Pre-PR, the `stat.size == 0 or stat.kind != .file` guard silently treated a directory-as-env-file (issue #3670) as an empty source and moved on. My refactor inverted that test: non-regular files now *enter* the read-until-EOF loop, which means a directory fd would be passed to read(), get EISDIR, and surface as 'IsDir error loading .env file' on stderr under `--loglevel=info`. Add an explicit `bun.S.ISDIR` check that returns null (empty source) before the read loop. The #3670 regression test (".env in a folder doesn't throw an error") still passes. Reported by claude[bot].
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/dotenv/env_loader.zig`:
- Around line 811-818: The null terminator is being written at the probed size
variable pos instead of the actual bytes read, so change the sentinel assignment
from buf[pos] = 0 to buf[amount_read] = 0 (and ensure the allocation still has
room for amount_read + 1); update the same change in the second occurrence
(lines 866-873). Locate the read path using bun.sys.readAll, the local variables
buf, pos and amount_read, and write the trailing zero at buf[amount_read] to
ensure the buffer is null-terminated at the real read length.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 0ddeb553-b747-433c-a879-138a076a5a7a
📒 Files selected for processing (1)
src/dotenv/env_loader.zig
If the file shrinks between getFileSize/fstat and readAll, amount_read can be less than pos/size. The helper's doc contract says the null byte sits at `buf[amount_read]`, but the code was writing it at buf[pos]/buf[size], leaving buf[amount_read] uninitialized on a partial read. Callers only slice `buf[0..amount_read]` so this wasn't a functional bug for the parser, but it violates the documented sentinel contract. Reported by coderabbit.
Fixes #30520
Problem
bun --env-file=<FIFO>silently loaded zero variables when a writerwas attached. Reproduction:
This broke 1Password's local-env-file
integration, which exposes secrets through a FIFO so they never touch
disk.
cat,dotenv, andnode:fs.readFileSyncall read the sameFIFO correctly.
Cause
src/dotenv/env_loader.zig(bothloadEnvFileandloadEnvFileDynamic)sized its read buffer from
stat.sizeand short-circuited whenstat.size == 0 or stat.kind != .file:FIFOs always report
st_size == 0andstat.kind == .named_pipe, sothe loader recorded the path as 'loaded' with empty content and
returned without reading anything.
Fix
Extract the read logic into
readEnvFileContents. For non-regularfiles (FIFOs, sockets, character devices), read until EOF into a
growing
ArrayList(initial capacity 4 KiB, reserves 4 KiB of unusedspace before each read) instead of preallocating from
stat.size.Regular files keep the existing fast path that preallocates from
stat.size.Both
loadEnvFile(default.envdiscovery) andloadEnvFileDynamic(
--env-file=) share the helper.Verification
test/cli/run/env.test.tsadds two tests inside the--env-filedescribe block, gated on
isPosix:reads variables from a FIFO with a writer attached— spawns awriter that writes a single
BUNTEST_FIFO=hello-fifoline, readsit back via
--env-file, and checks the value surfaces inprocess.env.reads FIFO content larger than the initial read buffer— writes500 entries (~16 KiB) through the FIFO to exercise the grow loop,
then spot-checks the first and last entry.
Both tests fail with
git stash push -- src/and pass with the fixapplied (
stash pop).bun run zig:check-allis clean across Linux x86_64/aarch64, Windowsx86_64, and macOS. The Windows code path is unchanged (keeps using
file.getEndPos()).