dotenv: stop panicking on nested ${...} inside ${VAR:-default} - #36199
Conversation
|
Warning Review limit reached
Next review available in: 12 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
Comment |
|
Found 1 issue this PR may fix:
🤖 Generated with Claude Code |
|
Updated 8:04 PM PT - Jul 28th, 2026
✅ @robobun, your commit ae28007715c10f83271ab44d205f72b167ea62fd passed in 🧪 To try this PR locally: bunx bun-pr 36199That installs a local version of the PR into your bun-36199 --bun |
|
This PR may be a duplicate of:
🤖 Generated with Claude Code |
|
Supersedes #32413 (same fix from an earlier run, now conflicting on main; closed). |
There was a problem hiding this comment.
I didn't find any bugs, but this is a full rewrite of expand_value — startup-path code that runs on every .env load — and it changes semantics beyond the panic fix, so it's worth a human look.
Checked: the depth-counted ${...} matcher can't produce inverted slices; \$, $IDENT, and the existing .env value expansion / .env special characters / .env escaped dollar sign tests still trace to the same output; recursion is bounded at 200.
Ruled out: parse_quoted's per-quote-type \ handling composes correctly with the new scanner (backtick \$ still yields $); the changed flag returning None for pure-literal / unterminated input keeps the original bytes.
Behaviour changes beyond the crash fix that a maintainer should sign off on: ${KEY<junk>} (e.g. ${A-x}) now falls through as the whole literal instead of partially consuming; $$ now yields a literal $ (old: both consumed); a bare trailing \$ now becomes $ (old: kept as \$).
Extended reasoning...
Overview
Rewrites Parser::expand_value in src/dotenv/env_loader.rs from a right-to-left $ scan with forward :- default lookup into a single left-to-right depth-counting pass (expand_into), fixing a slice-inversion panic on nested ${...} inside ${VAR:-default}. Adds a 17-case table test in test/cli/run/env.test.ts. The old function is entirely replaced (~70 lines removed, ~110 added).
Security risks
Low. Input is a local .env file already trusted for env-var injection. The new code eliminates a user-reachable panic (which is itself a robustness fix), bounds recursion at depth 200, and every slice index is derived from a matched brace or a monotone cursor — no arithmetic on untrusted lengths.
Level of scrutiny
High. This code runs during startup for every Bun invocation in a directory containing a .env, so any regression is very visible. It is not a minimal patch: the algorithm is replaced wholesale and carries several intentional semantic changes beyond stopping the crash — ${KEY<non-:- junk>} now round-trips as literal text (previously it partially consumed and emitted the tail), $$ now yields a literal $ (previously both dollars were consumed as an empty-key lookup), a lone \$ at end-of-value now becomes $ (previously left as \$), and :- defaults are now recursively expanded (previously literal, though the nested case panicked so there is no working old behaviour to preserve). These are all defensible moves toward dotenv-expand / docker-compose semantics, but they are user-observable changes to a widely-used feature and should be confirmed by a maintainer.
Other factors
The new test is thorough (nested, quoted, prefixed, bare-$, empty-key, unterminated, backslash, non-:-), and I traced the pre-existing expansion tests (.env value expansion, .env escaped dollar sign, .env special characters 1 (issue #2823)) through the new scanner and they still hold. The PR description notes tests were not run locally ("no test proof … deferring to CI"), and CI was still building at the time of the last timeline update. Two comment-cop bot notes fired on earlier revisions and are marked resolved; the current doc comment on expand_into is concise. Given the blast radius and the deliberate semantic shifts, deferring to a human.
|
The three behaviour changes the review bot flagged are intentional and follow the requested scanner spec:
The existing |
…t panic
A .env line like `HOST=${HOST:-${DEFAULT_HOST}}` crashed every bun
invocation in that directory with "panic: slice index starts at N but
ends at M" before any user code ran.
expand_value walks the value right-to-left, processing each `$` and
recording `last` as the start of the most recently handled reference.
The inner `${DEFAULT_HOST}` sets `last` to its `$`, then the outer
`${HOST:-...}` scans its key / `:-` default / closing `}` forward
bounded only by value.len(), advancing `end` past `last` and panicking
on the `value[end..last]` splice.
Bound every forward scan in that branch by `last` instead of
value.len(). For the very first (rightmost) `$`, last == value.len()
so nothing changes; for an outer reference whose default nests another
`${...}`, the scan now stops at the inner boundary and the loader
completes instead of aborting.
This does not add full recursive default expansion: the outer's closing
`}` was already spliced as literal by the inner pass, so it leaks
through (`localhost}`). The test pins that behaviour so a later
recursive implementation has an obvious place to tighten.
The previous right-to-left `$` scan advanced a forward `:-` default
scan past the already-consumed `last` boundary whenever the default
clause contained another reference (`${A:-${B}}`, `${A:-$B}`,
`${A:-$}`, ...), inverting the `value[end..last]` splice and
panicking the loader before any user code ran.
Replace it with a single left-to-right pass:
- `${...}` finds its closing brace by depth counting (`${` opens,
`}` closes, `\x` is skipped), so every slice is bounded by a
matched index.
- Bare `${KEY}` resolves to the key's value or empty.
- `${KEY:-default}` resolves to the key's value if set, otherwise the
default is itself expanded recursively, so `${A:-${B:-c}}` composes.
- Unterminated `${...` and junk after the key (`${A-x}`, `${A.x}`)
fall through as the literal text; the loader never aborts on shape.
- `$IDENT` and `\$` keep their existing behaviour.
The test covers the nested, quoted, prefixed, bare-$, empty-key,
unterminated, and non-`:-` forms that previously crashed or leaked a
stray `}`.
47f9469 to
70186d5
Compare
There was a problem hiding this comment.
Thanks for addressing both earlier nits (issue reference + using tempDir) — those look good now. I'm deferring to a human reviewer because this is a full rewrite of .env value expansion that ships three intentional behaviour changes beyond the panic fix ($$ stays literal, ${KEY<junk>} round-trips, trailing \$ handling), and .env parsing runs on every bun invocation — worth a maintainer sign-off on the new semantics.
Checked: depth-counted brace matching keeps every slice bounded (no inverted-range panic possible); unterminated forms fall through without setting changed, so the original value is kept unchanged; the recursive default expansion is bounded by depth < 200 and each level consumes a strict substring; the pre-existing .env value expansion, .env escaped dollar sign, and .env special characters 1 (issue #2823) tests trace correctly through the new scanner.
Extended reasoning...
Overview
Rewrites Parser::expand_value / new expand_into in src/dotenv/env_loader.rs from a right-to-left $ scan (which panicked with an inverted slice on nested ${...} inside a :- default) to a left-to-right depth-counting scanner. Adds a 17-case regression table in test/cli/run/env.test.ts. Fixes #32411.
Security risks
None identified. Input is the user's own .env file, output goes into process.env. The only untrusted-input concern would be pathological nesting causing stack overflow; the depth < 200 cap plus the fact that each recursion operates on a strict substring bounds this. All indexing is via value.get() / matched-brace positions, so no out-of-bounds slicing.
Level of scrutiny
Medium-high. This is not a minimal fix — it's a ~100-line rewrite of the expansion scanner with three documented behaviour changes beyond the panic fix (per the author's own comment on the PR): ${KEY<junk>} now round-trips as literal instead of emitting a stray } tail; $ followed by a non-{/non-ident char is now literal (so $$ stays $$ rather than collapsing the first $ via empty-key lookup); and \$ at end-of-value is now handled. .env parsing runs on essentially every bun invocation in a project directory, so even reasonable-looking semantic changes have wide blast radius and deserve a maintainer's eye — particularly the $$ change, which could surprise users who relied on the old collapse.
Other factors
- Both nits from the previous review pass were addressed exactly as suggested in f17e668 (test now cites #32411 in name + URL comment; uses
using dir = tempDir(...)withString(dir)for cwd). - CI green on the last reported build.
- I traced the three unchanged existing expansion tests (
.env value expansion,.env escaped dollar sign,.env special characters 1) through the new code and they produce the asserted outputs. - The
changedflag correctly staysfalsefor pure-unterminated inputs so the original box is retained without an unnecessary re-allocation; when any prior segment setchanged, the unterminated tail is still copied intooutand the modified buffer is returned. - No bugs found by the bug-hunting system this run.
There was a problem hiding this comment.
Both earlier nits are addressed — thanks. No bugs found on this pass, but this is a full rewrite of the .env value expander (runs on every startup) that intentionally changes three user-visible behaviors (${KEY<junk>} now literal, $$ now literal, trailing \$ now stripped). Those look reasonable to me but deserve a human sign-off.
What was reviewed
- Depth-counting close-brace scan: every slice is bounded by a matched index or falls through to literal — no inverted-range panic reachable.
changedflag: unterminated${...correctly leaves it unset so the original value is retained; verified against theNSTD_UNTERM/NSTD_UNTERM2cases.- Recursion depth cap (200) prevents stack overflow on adversarial nesting; overflow path emits the default literally.
- Existing
.env value expansion,.env escaped dollar sign, and issue #2823 tests are unchanged and their expected outputs are preserved by the new scanner.
Extended reasoning...
Overview
This PR replaces Parser::expand_value in src/dotenv/env_loader.rs — the routine that expands $NAME / ${NAME} / ${NAME:-default} references in .env values — with a new left-to-right depth-counting implementation (expand_into). The old right-to-left scan panicked with an inverted slice range whenever a :- default clause itself contained a $ reference. A 17-case regression table is added to test/cli/run/env.test.ts.
Security risks
None material. Input is a local .env file the user controls; the worst pre-fix outcome was a startup panic. The new code has a hard recursion depth cap (200) and no unbounded allocation driven by input structure, so an adversarial .env cannot blow the stack or OOM via nesting. All indexing is either value.get() or bounded by a matched close index / value.len().
Level of scrutiny
Medium-high. This is not a mechanical fix — it is a ~100-line parser rewrite of a routine that runs on every bun invocation in any directory containing a .env file, and it deliberately changes three edge-case behaviors (documented by the author in a PR comment): ${KEY<junk>} now round-trips literally instead of emitting a mangled tail; a $ followed by neither { nor an identifier is now literal (so $$ → $$, previously $); and \$ at end-of-value now strips the backslash. Each is defensible and arguably a strict improvement, but they are user-visible semantic changes to .env parsing that a maintainer should explicitly accept rather than have a bot approve.
Other factors
- My two prior nits (test comment narration,
tempDirWithFiles→using tempDir) were addressed in f17e668 and the threads are resolved; the current diff reflects both fixes. - The bug-hunting system found nothing on this revision.
- Test coverage is thorough for the new behavior (nested, quoted, prefixed, bare-
$, empty-key, unterminated, backslash-in-default, non-:-junk), and the pre-existing expansion tests are untouched. - I traced the
changedreturn through the unterminated-${and pure-literal paths: when nothing expands,expand_valuereturnsNoneand the caller keeps the original bytes, so no accidental empty-string writes. - I did not attempt to compare against dotenv-expand / docker-compose semantics for the three changed edge cases; that judgment call is the main reason for deferring.
|
Status: ready for review.
|
* upstream/main: (422 commits) install: drop packages held only by optional-peer resolution slots from bun.lock (oven-sh#35681) Update mimalloc to the upstream dev3 (v3.4.3) sync (oven-sh#36431) compile(pe): ftruncate the Windows --compile output after writing (oven-sh#36430) Strong: back bun_jsc::Strong with StrongRootBlock; free AbortSignal.timeout at wrapper GC (oven-sh#35849) test(harness): replace toRun matcher with async bunRun + toSpawn (oven-sh#36424) test: measure memory via harness rss() instead of process.memoryUsage.rss() (oven-sh#36429) Deflake a few tests no-orphans(windows): allow CREATE_BREAKAWAY_FROM_JOB and set DIE_ON_UNHANDLED_EXCEPTION on the Job (oven-sh#36414) GarbageCollectionController: replace per-tick heap sampler with idle timer only (oven-sh#35356) exe_format(pe): write a valid OptionalHeader.CheckSum for --compile output (oven-sh#36383) FileSink: flush buffered bytes when process.exit() runs in the same tick as write() (oven-sh#36250) test(http): speed up and de-flake serve-async-stream-client-abort.test.ts (oven-sh#35919) test(20144): stop racing child startup against the 1s SIGKILL guard (oven-sh#34166) test(no-orphans): skip fast-exit perl daemon test on macOS (oven-sh#36413) fs: return negative BigIntStats *Ns for pre-epoch timestamps (oven-sh#36187) event_loop: make DeferredTaskQueue::run tolerate re-entrant map mutation (oven-sh#32703) dotenv: stop panicking on nested `${...}` inside `${VAR:-default}` (oven-sh#36199) fetch: make the idle timer an absolute deadline for the response header block (oven-sh#36145) bundler: don't panic on unterminated naming template placeholders (oven-sh#36325) Buffer#indexOf/lastIndexOf: rare-byte SIMD filter with a Two-Way O(n+m) fallback (oven-sh#36420) ... # Conflicts: # src/jsc/bindings/BunDebugger.cpp
* upstream/main: (422 commits) install: drop packages held only by optional-peer resolution slots from bun.lock (oven-sh#35681) Update mimalloc to the upstream dev3 (v3.4.3) sync (oven-sh#36431) compile(pe): ftruncate the Windows --compile output after writing (oven-sh#36430) Strong: back bun_jsc::Strong with StrongRootBlock; free AbortSignal.timeout at wrapper GC (oven-sh#35849) test(harness): replace toRun matcher with async bunRun + toSpawn (oven-sh#36424) test: measure memory via harness rss() instead of process.memoryUsage.rss() (oven-sh#36429) Deflake a few tests no-orphans(windows): allow CREATE_BREAKAWAY_FROM_JOB and set DIE_ON_UNHANDLED_EXCEPTION on the Job (oven-sh#36414) GarbageCollectionController: replace per-tick heap sampler with idle timer only (oven-sh#35356) exe_format(pe): write a valid OptionalHeader.CheckSum for --compile output (oven-sh#36383) FileSink: flush buffered bytes when process.exit() runs in the same tick as write() (oven-sh#36250) test(http): speed up and de-flake serve-async-stream-client-abort.test.ts (oven-sh#35919) test(20144): stop racing child startup against the 1s SIGKILL guard (oven-sh#34166) test(no-orphans): skip fast-exit perl daemon test on macOS (oven-sh#36413) fs: return negative BigIntStats *Ns for pre-epoch timestamps (oven-sh#36187) event_loop: make DeferredTaskQueue::run tolerate re-entrant map mutation (oven-sh#32703) dotenv: stop panicking on nested `${...}` inside `${VAR:-default}` (oven-sh#36199) fetch: make the idle timer an absolute deadline for the response header block (oven-sh#36145) bundler: don't panic on unterminated naming template placeholders (oven-sh#36325) Buffer#indexOf/lastIndexOf: rare-byte SIMD filter with a Two-Way O(n+m) fallback (oven-sh#36420) ...
* upstream/main: (422 commits) install: drop packages held only by optional-peer resolution slots from bun.lock (oven-sh#35681) Update mimalloc to the upstream dev3 (v3.4.3) sync (oven-sh#36431) compile(pe): ftruncate the Windows --compile output after writing (oven-sh#36430) Strong: back bun_jsc::Strong with StrongRootBlock; free AbortSignal.timeout at wrapper GC (oven-sh#35849) test(harness): replace toRun matcher with async bunRun + toSpawn (oven-sh#36424) test: measure memory via harness rss() instead of process.memoryUsage.rss() (oven-sh#36429) Deflake a few tests no-orphans(windows): allow CREATE_BREAKAWAY_FROM_JOB and set DIE_ON_UNHANDLED_EXCEPTION on the Job (oven-sh#36414) GarbageCollectionController: replace per-tick heap sampler with idle timer only (oven-sh#35356) exe_format(pe): write a valid OptionalHeader.CheckSum for --compile output (oven-sh#36383) FileSink: flush buffered bytes when process.exit() runs in the same tick as write() (oven-sh#36250) test(http): speed up and de-flake serve-async-stream-client-abort.test.ts (oven-sh#35919) test(20144): stop racing child startup against the 1s SIGKILL guard (oven-sh#34166) test(no-orphans): skip fast-exit perl daemon test on macOS (oven-sh#36413) fs: return negative BigIntStats *Ns for pre-epoch timestamps (oven-sh#36187) event_loop: make DeferredTaskQueue::run tolerate re-entrant map mutation (oven-sh#32703) dotenv: stop panicking on nested `${...}` inside `${VAR:-default}` (oven-sh#36199) fetch: make the idle timer an absolute deadline for the response header block (oven-sh#36145) bundler: don't panic on unterminated naming template placeholders (oven-sh#36325) Buffer#indexOf/lastIndexOf: rare-byte SIMD filter with a Two-Way O(n+m) fallback (oven-sh#36420) ... # Conflicts: # src/js/internal/debugger.ts
* upstream/main: (422 commits) install: drop packages held only by optional-peer resolution slots from bun.lock (oven-sh#35681) Update mimalloc to the upstream dev3 (v3.4.3) sync (oven-sh#36431) compile(pe): ftruncate the Windows --compile output after writing (oven-sh#36430) Strong: back bun_jsc::Strong with StrongRootBlock; free AbortSignal.timeout at wrapper GC (oven-sh#35849) test(harness): replace toRun matcher with async bunRun + toSpawn (oven-sh#36424) test: measure memory via harness rss() instead of process.memoryUsage.rss() (oven-sh#36429) Deflake a few tests no-orphans(windows): allow CREATE_BREAKAWAY_FROM_JOB and set DIE_ON_UNHANDLED_EXCEPTION on the Job (oven-sh#36414) GarbageCollectionController: replace per-tick heap sampler with idle timer only (oven-sh#35356) exe_format(pe): write a valid OptionalHeader.CheckSum for --compile output (oven-sh#36383) FileSink: flush buffered bytes when process.exit() runs in the same tick as write() (oven-sh#36250) test(http): speed up and de-flake serve-async-stream-client-abort.test.ts (oven-sh#35919) test(20144): stop racing child startup against the 1s SIGKILL guard (oven-sh#34166) test(no-orphans): skip fast-exit perl daemon test on macOS (oven-sh#36413) fs: return negative BigIntStats *Ns for pre-epoch timestamps (oven-sh#36187) event_loop: make DeferredTaskQueue::run tolerate re-entrant map mutation (oven-sh#32703) dotenv: stop panicking on nested `${...}` inside `${VAR:-default}` (oven-sh#36199) fetch: make the idle timer an absolute deadline for the response header block (oven-sh#36145) bundler: don't panic on unterminated naming template placeholders (oven-sh#36325) Buffer#indexOf/lastIndexOf: rare-byte SIMD filter with a Two-Way O(n+m) fallback (oven-sh#36420) ...
A
.envline likecrashed every
buninvocation in that directory withbefore any user code ran (rc 134, core). Any
:-default clause containing a$triggered it:${A:-$B},${A:-${B}},${A:-$},${A:-${B:-c}},$${A:-${B}},${:-${B}}, quoted/prefixed variants, and unterminated${A:-${.Cause
Parser::expand_valueinsrc/dotenv/env_loader.rsscanned the value right-to-left for$, trackinglastas the start of the most recently processed (nearest-to-the-right) reference. When the outer${A:-...}was reached, its:-default clause was then scanned forward bounded only byvalue.len(), so it walked straight across the inner reference that begins atlast, landed on the inner's closing}, and leftend > last. The subsequentvalue[end..last]splice inverted and panicked.Fix
Replace the reverse
$scan + forward default scan with one left-to-right depth-counting pass:${...}finds its closing}by depth (${opens,}closes,\xis skipped), so every slice is bounded by a matched index and cannot invert.${KEY}resolves to the key's value or empty.${KEY:-default}resolves to the key's value if set; otherwise the default clause is itself expanded recursively, so${A:-${B:-c}}composes the way dotenv-expand / docker-compose users expect.${...and junk after the key (${A-x},${A.x}) fall through as the literal input; the loader never aborts on shape.$IDENTand\$keep their existing behaviour.Verification
test/cli/run/env.test.tsgains a table covering the nested, quoted, prefixed, bare-$, empty-key, unterminated, backslash, and non-:-forms. The existing.env value expansiontest is unchanged and still passes.no test proof · iteration 1 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/cli/run/env.test.ts
Fixes #32411