Skip to content

glob: expand brace groups that span path separators when scanning - #32599

Open
robobun wants to merge 8 commits into
mainfrom
farm/37a5746d/glob-scan-brace-separators
Open

robobun wants to merge 8 commits into
mainfrom
farm/37a5746d/glob-scan-brace-separators

Conversation

@robobun

@robobun robobun commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator

Problem

Bun.Glob.scan() / scanSync() return an empty result for brace patterns where an alternative contains a path separator, e.g. svc/{src/env.ts,env.ts}. match() handles the same pattern correctly, and so do Node's fs.glob and fast-glob, so the scanner is inconsistent with both.

const g = new Bun.Glob("svc/{src/env.ts,env.ts}");
[...g.scanSync({ cwd })];   // []     (wrong, should find both files)
g.match("svc/src/env.ts");  // true
g.match("svc/env.ts");      // true

Fixes #32596.

Cause

The scanner (src/glob/GlobWalker.rs) models a pattern as one component per directory level and splits the pattern on every path separator, including separators inside a brace group. svc/{src/env.ts,env.ts} is therefore cut into the components svc, {src and env.ts,env.ts}, which match no real directory layout, so the walk yields nothing. A brace alternative containing a separator spans multiple levels with different depths (src/env.ts is two levels, env.ts is one), which the one-component-per-level model cannot represent. match() is unaffected because it evaluates braces inline over the whole string (src/glob/matcher.rs).

Fix

Expand a pattern whose braces contain a path separator into separate brace-free patterns, the same set of strings match() evaluates the braces against, and walk each in turn. The walker rebuilds its components per expansion and dedupes results through the existing matched_paths set, so overlapping alternatives collapse to one result. Expansion honors backslash escapes, [...] bracket classes and nested braces, and is bounded (10k patterns, depth 32) so adversarial patterns cannot blow up. Patterns with no separator inside braces ({a,b}/c, src/{x,y}.ts) keep the existing single-pass behavior unchanged.

The expansion drives the shared walker Iterator, so scan, scanSync, and the other scan entry points (shell globbing, workspaces, --filter) all get the fix.

Verification

Added regression coverage in test/js/bun/glob/scan.test.ts (both scan and scanSync): the reported pattern, alternatives of differing depth, a globstar inside a brace alternative, dedup of overlapping alternatives, and a scan/match agreement check. Unit tests for the expansion and detection helpers live in src/glob/GlobWalker.rs.

$ bun bd test test/js/bun/glob/scan.test.ts -t "brace patterns containing path separators"
 11 pass   0 fail

All 11 fail on main (each returns []) and pass with this change.

Note

#25789 took the same approach earlier, but it patches src/glob/GlobWalker.zig, the original implementation that has since been ported to Rust and is no longer compiled, and it currently has merge conflicts. This change implements the same idea against the live Rust walker and covers the shared iterator that every scan path uses.

Bun.Glob.scan() returned an empty result for brace patterns where an
alternative contains a path separator (e.g. `svc/{src/env.ts,env.ts}`),
even though match() handled the same pattern correctly.

The scanner models a pattern as one component per directory level and
split the pattern on every separator, including separators inside a
brace group. `svc/{src/env.ts,env.ts}` was therefore cut into the bogus
components `svc`, `{src` and `env.ts,env.ts}` and matched nothing. A
brace alternative containing a separator spans multiple levels, which
the one-component-per-level model cannot represent.

Expand such patterns into separate brace-free patterns (the same set of
strings match() evaluates the braces against) and walk each in turn,
rebuilding the components per expansion and deduping results through the
existing matched_paths set. Expansion honors backslash escapes, `[...]`
bracket classes and nested braces, and is bounded so adversarial
patterns cannot blow up. Patterns with no separator inside braces keep
the existing single-pass behavior.

The expansion drives the shared walker Iterator, so scan, scanSync and
the other scan entry points (shell globbing, workspaces, --filter) all
benefit.
@robobun

robobun commented Jun 22, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 3:45 PM PT - Jun 22nd, 2026

❌ @robobun, your commit 7ee1584 has 4 failures in Build #63963 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 32599

That installs a local version of the PR into your bun-32599 executable, so you can run:

bun-32599 --bun

@github-actions

Copy link
Copy Markdown
Contributor

Found 1 issue this PR may fix:

  1. globSync(‘{*/*}’) always returns empty array #24000 - globSync('{*/*}') returns empty array because the brace group contains a path separator — the exact bug class this PR fixes

If this is helpful, copy the block below into the PR description to auto-close this issue on merge.

Fixes #24000

🤖 Generated with Claude Code

@github-actions

Copy link
Copy Markdown
Contributor

This PR may be a duplicate of:

  1. fix(glob): expand braces containing path separators before walking #25789 - Same fix: expands brace groups containing path separators before glob walking, targeting the same GlobWalker code area

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Jun 22, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

GlobWalker gains brace-expansion support for patterns where brace alternatives contain path separators (e.g., svc/{src/env.ts,env.ts}). Such patterns are expanded into multiple brace-free patterns at init time; the iterator traverses each expansion sequentially, deduplicating matches across expansions. New Rust unit tests and TypeScript integration tests cover the behavior.

Changes

GlobWalker brace expansion for path-separator alternatives

Layer / File(s) Summary
GlobWalker brace expansion fields and init
src/glob/GlobWalker.rs
Adds brace_expansions: Vec<Box<[u8]>> and expansion_cursor: usize to GlobWalker. init_with_cwd detects brace groups with path separators, calls expand_braces to populate brace_expansions, and widens the walk early-exit guard to check both pattern_components and brace_expansions.
Brace-expansion parsing and expansion subsystem
src/glob/GlobWalker.rs
New private functions implement separator detection (respecting bracket classes and backslash escapes on non-Windows) and recursive brace expansion bounded by depth and pattern-count constants, discarding patterns that collapse to empty after expansion.
Iterator rebuild and multi-expansion traversal
src/glob/GlobWalker.rs
Iterator::init calls rebuild_components_for_current_expansion when expansions are present. Root-open error handling treats ENOENT/ENOTDIR as yielding no matches for that alternative. try_advance_to_next_expansion resets iterator state and re-runs init for the next expansion. Iterator::next advances through expansions on work-stack exhaustion instead of returning None immediately.
Unit and integration tests
src/glob/GlobWalker.rs, test/js/bun/glob/scan.test.ts
Rust tests verify separator detection and expansion correctness for nested braces, cartesian products, bracket-class comma handling, and empty-alternative dropping. TypeScript tests assert Glob.scan/Glob.scanSync return correct sorted entries for separator-containing brace patterns, cross-check results with glob.match(), and validate behavior on Windows with absolute-literal expansions when one alternative's root is missing.

Possibly related PRs

  • oven-sh/bun#31367: The main PR changes GlobWalker to treat ENOENT/ENOTDIR during brace-alternative root opens as "no matches" (avoiding aborting the scan), and the retrieved PR changes shell Expansion::on_glob_walk_done to stop throwing on glob-walker failures—both directly adjust how missing-dir glob-walk errors are handled end-to-end.
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The PR title 'glob: expand brace groups that span path separators when scanning' clearly and concisely summarizes the main change: enabling brace pattern expansion for separators during glob scanning operations.
Description check ✅ Passed The PR description comprehensively covers both required template sections: a detailed Problem section explaining the issue with reproduction code, and a Verification section documenting test coverage and command results.
Linked Issues check ✅ Passed The PR implementation fully addresses issue #32596: brace patterns with path separators now expand correctly, allowing scan()/scanSync() to find matching files consistent with match() behavior, with comprehensive test coverage added.
Out of Scope Changes check ✅ Passed All code changes are directly scoped to the brace-separator expansion objective: GlobWalker.rs adds brace expansion logic, and scan.test.ts adds focused regression tests for the specific issue without unrelated modifications.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/glob/GlobWalker.rs`:
- Around line 2225-2248: In the brace_group_spans_separator function, the
backslash escape handling at the match arm for b'\\' is being processed before
checking if the backslash is a native path separator. On Windows, backslash is a
native separator and should not be unconditionally treated as an escape
character. Modify the escape-skipping logic to either only apply on non-Windows
platforms or check if the backslash is a native separator first before
incrementing i by 2 to skip it. Ensure that on Windows, a backslash can still be
classified as a separator through the bun_core::path_sep::is_sep_native check.
- Around line 585-606: The absolute-literal fast path in the init() method can
set IterState::Matched(path) without inserting the path into matched_paths,
causing duplicate paths to be emitted when processing overlapping brace
expansions like /tmp/{a/b,a/b}. In the iteration logic where IterState::Matched
paths are returned to consumers (likely in the walk() method or similar
iteration handler), add a check against matched_paths.contains_key() before
returning the path. If the path is not already recorded in matched_paths, insert
it first, ensuring that all matched paths are properly deduplicated across
expansion boundaries regardless of whether they came from the fast path or
regular path in init().
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: dcc8fd6c-05cf-4c94-a31a-cf5807cb7007

📥 Commits

Reviewing files that changed from the base of the PR and between 85559eb and 7b303bb.

📒 Files selected for processing (2)
  • src/glob/GlobWalker.rs
  • test/js/bun/glob/scan.test.ts

Comment thread src/glob/GlobWalker.rs
Comment thread src/glob/GlobWalker.rs
Comment thread src/glob/GlobWalker.rs
Comment thread src/glob/GlobWalker.rs
…as a separator

Two fixes from review of the brace-expansion change:

- The no-special-syntax fast path in `Iterator::init` set `IterState::Matched`
  without recording the path in `matched_paths`. Because the iterator now
  re-runs `init` per brace expansion, an absolute pattern whose alternatives
  overlap (e.g. `/x/{a,a}` or `/x/{a,*}`) emitted the same file twice, and
  `walk` (which reads `matched_paths`) dropped fast-path matches entirely. Route
  the fast path through a new `record_full_match` helper that checks and inserts
  `matched_paths`, mirroring `prepare_matched_path`'s NUL handling so dedupe is
  exact in both SENTINEL modes. Absolute literals now surface from `scan` too.

- `brace_group_spans_separator` and `find_first_brace_group` consumed `\` as an
  escape on every platform, but `build_pattern_components` treats `\` as a path
  separator on Windows (`is_sep_native`). A pattern like `svc/{src\env.ts,b}`
  therefore skipped expansion yet still got split on the `\`. Gate the escape
  handling to non-Windows so a `\` inside a brace group triggers expansion on
  Windows, matching the splitter.

Adds scan/scanSync coverage for single-alternative wildcard groups spanning a
separator (the `{*/*}` shape) and for absolute-literal expansion surfacing/dedupe.
@robobun

robobun commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator Author

Notes on the two automated findings above:

#24000 (globSync('{*/*}')) is a separate code path, so this PR does not close it. node:fs glob/globSync have their own implementation in src/js/internal/fs/glob.ts (vendored minimatch + Node's fs.readdir); they do not use the Bun.Glob / GlobWalker scanner changed here. This PR does fix the same symptom on the Bun.Glob.scan/scanSync path, e.g. new Bun.Glob("{*/*}").scanSync() now returns results, covered by the added {src/*.ts} and pkg/{a/*/*.ts} tests. Fixing node:fs globSync would be a separate change in the minimatch layer, so I'm leaving #24000 open.

#25789 takes the same approach (expand brace groups containing a separator before walking), but it targets src/glob/GlobWalker.zig, the original implementation that has since been ported to Rust and is no longer compiled, and it currently has merge conflicts. This PR implements the idea against the live Rust walker (src/glob/GlobWalker.rs) and drives it through the shared iterator, so scan, scanSync, shell globbing, workspaces, and --filter all benefit.

Comment thread src/glob/GlobWalker.rs
The earlier dedupe change (record_full_match) routed Iterator::init's
no-special-syntax fast path through matched_paths. That surfaced absolute
literal matches scan/scanSync had always dropped, including directories that
onlyFiles excludes: the `../.` matrix case started returning `[""]` instead of
`[]`, diverging from the fast-glob oracle. Revert it and restore the original
fast path. The double-emit it guarded against only reached next() consumers for
absolute brace patterns, which shell (its own brace expansion) and
workspace/--filter (relative patterns) do not produce.

Separately, walking each brace expansion re-runs init with that expansion's own
root, so an absolute alternative with a missing prefix (e.g.
`{/missing/*.ts,ok/*.ts}`) threw ENOENT and aborted the whole scan, dependent on
alternative order. Brace alternatives are an unordered set, so tolerate
ENOENT/ENOTDIR from the per-expansion root open (yield nothing for that
alternative) when brace_expansions is non-empty; single patterns keep the
original error so a bad cwd still surfaces.

Keeps the Windows backslash-in-braces fix. Adds a scan test for the missing-root
case and drops the now-invalid absolute-surfacing test.
Comment thread src/glob/GlobWalker.rs Outdated
…pans

`brace_group_spans_separator` gated its separator check on `!in_brackets`, but
`build_pattern_components` splits on a path separator regardless of bracket
state. For a degenerate pattern like `{x,a[/]b}` the detector returned false, so
expansion was skipped and the splitter cut the group into `{x,a[` / `]b}`,
losing the valid `x` alternative. Drop the `!in_brackets` guard from the
separator arm (keeping it on the brace/comma arms) so the detector mirrors the
splitter and the group's other alternatives expand instead of being mis-split.
Comment thread src/glob/GlobWalker.rs Outdated
Comment thread src/glob/GlobWalker.rs
…caped separators

Two follow-ups from review of the brace-expansion change:

- The ENOENT/ENOTDIR tolerance for a missing per-expansion root also fired for
  relative brace patterns, whose root is the shared user `cwd`, so
  `{a/b,c/d}` with a non-existent `cwd` silently returned `[]` instead of
  throwing. Gate it on `was_absolute` so only an absolute alternative's own
  literal prefix is tolerated; a bad `cwd` (and single patterns) still surface
  the error.

- `brace_group_spans_separator` consumed a backslash-escaped separator as an
  escape, but `build_pattern_components` splits on a separator regardless of a
  preceding backslash, so `svc/{src\/env.ts,env.ts}` skipped expansion and got
  mis-split. A new `escape_advance` helper stops the backslash arm from
  consuming a following separator (detector and both `find_first_brace_group`
  loops), so the detector mirrors the splitter and sibling alternatives expand.

Adds a scan test that a missing cwd still throws for a relative brace pattern,
and unit tests for the escaped-separator case.
Comment thread src/glob/GlobWalker.rs Outdated
When a brace group expands to exactly one pattern (a single-alternative group
such as `{src/env.ts}`, or an unclosed group pushed verbatim), it is equivalent
to the brace-free pattern, so set it as `pattern` and fall through to the
single-pattern path instead of the per-expansion machinery. That makes `{X}`
byte-for-byte transparent to `X`: it shares the single-pattern fast path,
`onlyFiles` filtering, and error handling. Previously the 1-element
`brace_expansions` vec tripped the missing-root tolerance, so a single absolute
alternative with a missing root returned `[]` while the bare pattern threw.
`brace_expansions` is now only populated for genuine multi-alternative groups.

Adds a scan test that a single-alternative group throws on a missing absolute
root exactly like its bare form.
Comment thread src/glob/GlobWalker.rs Outdated
Comment thread src/glob/GlobWalker.rs
expand_braces filtered empty results at every recursion level and only
re-injected the empty case when a sub-expansion was entirely empty. An empty
alternative (`{X,}`) filling a whole sub-slot therefore lost its empty branch
even when an outer prefix/suffix would have made it a valid non-empty pattern:
`{a/b,c}{d,}` dropped `a/b` and `c`, and `x{a/b,{c,}}` dropped `x`, both of
which match() accepts.

Keep empty pieces through the recursion (an empty input now yields one empty
string, so the per-level empty-vec fixups are gone) and drop patterns that stay
empty once, in expand_braces. Adds unit tests and a scan case for a trailing
optional group.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All my earlier findings have been addressed and I have nothing further, but this reworks the glob walker's iterator state machine and root-open error semantics across scan/shell/workspaces/--filter, so it's worth a human pass before merge.

Extended reasoning...

Overview

This PR adds brace-expansion support to src/glob/GlobWalker.rs for patterns where a brace alternative contains a path separator (e.g. svc/{src/env.ts,env.ts}). It introduces ~200 lines of new parsing/expansion logic (brace_group_spans_separator, find_first_brace_group, expand_braces), two new walker fields (brace_expansions, expansion_cursor), and modifies Iterator::init, Iterator::next, and walk() to re-run init per expansion and advance through expansions on work-stack exhaustion. It also changes root-open error handling to tolerate ENOENT/ENOTDIR for absolute brace alternatives. Tests added in both Rust unit tests and test/js/bun/glob/scan.test.ts.

Security risks

None identified. The new parser is bounded (10k expansions, depth 32) so adversarial patterns can't blow up memory or stack. No new filesystem operations beyond what the existing walker already does; no auth/crypto/permissions surface.

Level of scrutiny

High. This is core path-handling code shared by Bun.Glob.scan/scanSync, shell globbing, workspace resolution, and --filter. The change modifies the iterator state machine (re-running init() per expansion, which was previously single-shot) and alters error semantics (ENOENT/ENOTDIR now swallowed for absolute brace alternatives). Over the course of review I and CodeRabbit flagged seven distinct correctness issues across five fix-up commits — Windows \ handling, absolute-literal fast-path dedup (later reverted due to a snapshot regression), missing-root error propagation, the was_absolute gate, single-expansion transparency, escaped-separator detection, and empty-branch propagation. All have been addressed or consciously deferred (the unclosed-[ edge case), but the density of subtle interactions argues for a human reviewer to confirm the final shape.

Other factors

  • All prior inline comments from me are resolved; the bug-hunting pass on the current head found nothing new.
  • One nit (unclosed [ inside a brace alternative losing siblings) was intentionally left as-is with a reasonable justification — malformed input, heavier fix, documented behavior.
  • The visible CI status comment references an older commit (fb3cb0f) with failures; status for the current head (7ee1584) isn't shown in the thread.
  • The dedup-via-matched_paths fix for the absolute-literal fast path was reverted after it regressed an existing snapshot, leaving a known (narrow, documented) double-emit case for absolute brace patterns via next() consumers — a deliberate trade-off a human should be aware of.

@robobun

robobun commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator Author

CI red on 7ee1584 is unrelated flake, not this diff. The glob tests (test/js/bun/glob/scan.test.ts) pass on every lane; the failing jobs are all on macOS/Windows aarch64 lanes in unrelated subsystems:

  • test/integration/next-pages/test/dev-server-ssr-100.test.ts and next-build.test.ts (macOS aarch64): puppeteer chrome-headless-shell download failure (infra; the same download failure also hit build 63929).
  • test/js/bun/s3/s3.test.ts R2 large-file upload (macOS aarch64): 15s network timeout.
  • test/cli/update_interactive_install.test.ts (Windows aarch64): package install exit code 1, flaky (ran with 1 retry).

None touch the glob walker. For non-brace patterns (what workspace resolution and --filter use) this change is behaviorally identical to before: brace_expansions stays empty, so the expansion path, the ENOENT/ENOTDIR root-open tolerance (gated on !brace_expansions.is_empty()), and the multi-expansion advance are all no-ops. The diff is green on the lanes it affects and both bot reviews are clean, so this is ready for a maintainer pass.

@robobun

robobun commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator Author

Independently reproduced and landed on the same fix on current main (332f744): farm/af47808c/glob-scan-brace-slash. Same shape (expand only the /-spanning groups, cycle the Iterator, dedupe via matched_paths), slightly smaller (~180 LOC in GlobWalker.rs, 1024-pattern cap). Test suite overlaps; feel free to cherry-pick anything useful. scan.test.ts + match.test.ts: 234 pass / 0 fail.

@robobun

robobun commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

Triage note: #32596 was closed by the reporter after working around it downstream (archgate/cli#475), not by a change in Bun. The bug still reproduces on current main (f426a8e): new Glob("svc/{src/env.ts,env.ts}").scanSync({ cwd }) returns [] while match() returns true, and 19 of the 21 tests added here fail when applied on top of main. Leaving this open.

@robobun

robobun commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator Author

This bug was reported again with a brace group at the start of the pattern. The pattern {*.d.ts,vendor/**/*.d.ts} and the pattern {*,vendor/**/*}.d.ts return nothing from scanSync(). match() accepts the same files. This still reproduces on Bun 1.4.0 and on main at 152c561. The cause is the same, and the {src/*.ts} and pkg/{a/**/*.ts,b.ts} cases in this PR cover the shape.

The branch no longer merges with main. src/glob/GlobWalker.rs has three conflicts:

  • the new struct fields, next to pattern_components (main made it pub(crate))
  • the root open error path in the iterator (main now returns err.with_path(...))
  • the build_pattern_components(...) call in init

A rebase is needed before this can land.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bun.Glob.scan() silently returns empty results for brace patterns containing path separators

1 participant