Skip to content

glob: honour trailing '/' as 'directories only' throughout scan() - #34200

Open
robobun wants to merge 2 commits into
mainfrom
farm/06067573/glob-trailing-slash-globstar
Open

robobun wants to merge 2 commits into
mainfrom
farm/06067573/glob-trailing-slash-globstar

Conversation

@robobun

@robobun robobun commented Jul 15, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

A trailing / on a glob pattern is the standard idiom for "directories only" (bash globstar, fast-glob, node:fs.globSync). Bun.Glob().scanSync() violates this in three places:

import { Glob } from "bun";
import * as fs from "node:fs";

const R = "/tmp/glob-tsgs";
fs.rmSync(R, { recursive: true, force: true });
for (const d of ["d1/d2/d3", "e1"]) fs.mkdirSync(`${R}/${d}`, { recursive: true });
fs.writeFileSync(`${R}/d1/f.txt`, "x");
fs.writeFileSync(`${R}/foo`, "x");
const scan = p => [...new Glob(p).scanSync({ cwd: R, onlyFiles: false })].sort();

scan("**/");      // ["d1","e1"]          <- d1/d2 and d1/d2/d3 missing (globstar does not recurse)
scan("d1/**/");   // ["d1/d2"]            <- d1/d2/d3 missing
scan("**/foo/");  // ["foo"]              <- yields a *file*
scan("foo/");     // ["foo"]              <- yields a *file*
new Glob("d1/**/").match("d1/d2/d3/");    // true  <- scan can never yield a path its own matcher accepts
scan("d1/**");    // ["d1/d2","d1/d2/d3","d1/f.txt"]  <- control: no trailing slash recurses fine

Cause

build_pattern_components keeps the trailing separator inside component.len for the pattern-final component, and make_component classified the syntax hint on that raw len before detecting trailing_sep. So "**/" (len 3) missed the len == 2 => SyntaxHint::Double arm and stayed SyntaxHint::None; the walker only recurses on Double. Component::pattern_slice() already subtracts trailing_sep, which is why match() still sees bare ** and accepts deep paths.

Two adjacent paths carried the same defect:

  • match_pattern_file's **/X peek-ahead checked pattern.trailing_sep (the globstar's, always false) but not next_pattern.trailing_sep, so "**/foo/" and "**/*/" yielded regular files.
  • The Literal-tail statat() fast path in transition_to_dir_iter_state accepted S::ISREG without consulting trailing_sep, so "foo/" and "sub/foo/" yielded a regular file.

Fix

  • make_component now detects trailing_sep first and classifies the syntax hint on the same slice pattern_slice() returns (len - trailing_sep). "**/" classifies as SyntaxHint::Double and recurses like "**", while match_pattern_file's existing trailing_sep guard continues to exclude files.
  • match_pattern_file's !is_last branch additionally requires !next.trailing_sep.
  • The Literal-tail statat() fast path requires S::ISDIR(mode) when the component has a trailing separator.

As a consequence of classifying on the stripped slice, a bare "../" now agrees with ".." (both DotBack, both yield []); previously "../" classified as Literal and yielded [".."] via the statat fast path.

Verification

Added nine cases to the trailing directory separator describe block in test/js/bun/glob/scan.test.ts: **/, prefix/**/, scan/match agreement, async scan parity, **/foo/ on files and on directories, **/*/, foo/, sub/foo/, and */foo/. Eight fail with USE_SYSTEM_BUN=1 and all pass with this change; the ninth (**/foo/ on a directory) is a regression guard. Full scan.test.ts (203), match.test.ts (26), and node/fs/glob.test.ts (27) suites pass.


no test proof · iteration 3 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/bun/glob/scan.test.ts

@robobun

robobun commented Jul 15, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 7:37 PM PT - Jul 21st, 2026

❌ @robobun, your commit ceadce4 has 1 failures in Build #77295 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 34200

That installs a local version of the PR into your bun-34200 executable, so you can run:

bun-34200 --bun

@github-actions

Copy link
Copy Markdown
Contributor

Found 1 issue this PR may fix:

  1. Bun.Glob Cannot scan completely #21300 - Reports Bun.Glob('**').scanSync() failing to scan all files completely, which is the same recursive traversal bug this PR fixes by correcting globstar classification when a trailing separator is present.

If this is helpful, copy the block below into the PR description to auto-close this issue on merge.

Fixes #21300

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Jul 15, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Changes

The glob component parser now excludes trailing separators when classifying syntax and wildcard length, while tests cover recursive directory scanning with **/ patterns across sync, async, and match operations.

Globstar parsing and scanning

Layer / File(s) Summary
Component separator and syntax classification
src/glob/GlobWalker.rs
make_component derives trailing-separator state from component bytes, uses an effective length for syntax and wildcard classification, and updates ASCII handling for syntax hints.
Nested-directory globstar validation
test/js/bun/glob/scan.test.ts
Tests validate directory-only recursion, prefixed globstar patterns, Glob.match agreement, and sync/async scan equivalence.

Suggested reviewers: jarred-sumner

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely summarizes the main change: trailing slashes in glob scans are treated as directories-only.
Description check ✅ Passed The description covers the problem, fix, and verification, but it should use the repository's required headings.

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator Author

Re the find-issues suggestion: #21300 is unrelated. That report uses Glob('**') with no trailing slash (which already recurses correctly, as shown in the control case in this PR's repro) and describes incomplete results on Windows while the directory is being modified in VSCode. This PR only changes how the pattern-final **/ component is classified; it does not touch the directory-iteration path #21300 would implicate.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond the inline notes: checked that dropping saturating_sub(1) on last_idx is safe (the len == 0 early return already guards it), and that a bare-/ component (effective_len == 0) still classifies as Literal exactly as before.

Extended reasoning...

The two inline findings already cover the interesting collateral behavior (adjacent trailing-sep file admission is pre-existing; .//../ reclassification is a consistency nit). I additionally traced the moved block for underflow — the old code used saturating_sub(1) and the new code uses plain - 1, but make_component returns None on len == 0 before reaching it, so no underflow. I also traced the degenerate effective_len == 0 case (a component that is only /): comp_slice becomes empty, check_special_syntax is false, so it classifies as Literal — identical to the pre-PR path where comp_slice == b"/" also fell through to Literal. The WildcardFilepath branch still keys on component.len rather than effective_len, but for every trailing-slash input it reaches the same hint as before.

Comment thread src/glob/GlobWalker.rs Outdated
Comment thread src/glob/GlobWalker.rs
@robobun robobun changed the title glob: recognize a trailing-slash globstar as a globstar in scan() glob: honour trailing '/' as 'directories only' throughout scan() Jul 15, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — 95401e4 addresses both prior notes exactly as suggested, and the new excludes files when the final component carries a trailing separator block covers the matrix.

Checked the follow-up: !next.trailing_sep in the !is_last peek-ahead and the S::ISDIR gate on the statat fast path are both correct and minimal; next_pattern.unwrap() remains safe since !is_last implies idx+1 < len. Re-traced make_component for the effective_len == 0 case (bare /) — still Literal + trailing_sep, unchanged. */ reclassifying None→Single has no observable effect (same match_pattern_impl result, NT filter still matches everything).

Extended reasoning...

Overview

Three focused edits to src/glob/GlobWalker.rs: (1) make_component now detects trailing_sep up front and classifies on effective_len so **/ is SyntaxHint::Double; (2) match_pattern_file's !is_last peek-ahead adds && !next.trailing_sep; (3) the Literal-tail statat fast path gates on S::ISDIR(mode) when the component carries a trailing separator. Nine tests added to the existing trailing directory separator describe block in scan.test.ts.

Security risks

None. This is pattern-component classification and match gating inside the glob walker; no auth, crypto, or privilege boundaries. The change tightens behavior (a trailing / now excludes files on paths where it previously leaked them), which if anything reduces surprise.

Level of scrutiny

Moderate — glob traversal feeds the bundler, test runner, and install, so a misclassification could hide or over-match entries. That's why I traced every reclassified shape (**/, */, ./, ../, bare /) through both the walker and the Windows NT-filter path in the first pass, and re-checked the two follow-up hunks against the exact shapes I'd flagged. All check out; the ../ → DotBack side effect is confirmed intentional and consistency-restoring.

Other factors

I reviewed the first commit in detail and left two findings (one heads-up, one pre-existing same-class bug); both threads are now resolved and the author's fix in 95401e4 is exactly the two local edits I proposed, plus tests for each shape (**/foo/, **/*/, foo/, sub/foo/, */foo/, and the positive directory case). Tests use tempDir/using, are cross-platform via path.sep, and assert exact sorted arrays. The bug-hunting system found nothing on the follow-up commit. No CODEOWNERS entry for src/glob/.

@robobun

robobun commented Jul 15, 2026 •

Copy link
Copy Markdown
Collaborator Author

CI on ceadce4 (build 77295, post-rebase): the glob suites pass on every lane. Remaining red is unrelated to this diff:

  • test/cli/run/no-orphans.test.ts timed out on darwin 14 x64 in the perl daemon-reap case; the same test is tagged [flaky] on debian aarch64 in this build. Reported for main-break triage.
  • Everything else (dns-tcp-bidirectional-poll, test-http-server-connections-checking-leak, in-process-cron, es-module-lexer, test-repl-close, bun-server) is tagged [flaky] and passed on retry.

This diff touches only src/glob/GlobWalker.rs and test/js/bun/glob/scan.test.ts; none of the failing tests intersect that code. Ready for review.

@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

@robobun fix conflicts and push

robobun added 2 commits July 22, 2026 00:41
make_component() classified the syntax hint before detecting the
trailing separator, so the pattern-final component "**/" (len 3) missed
the len == 2 => SyntaxHint::Double arm and was left as a generic
one-segment pattern. The walker only recurses on SyntaxHint::Double, so
scan("**/") and scan("prefix/**/") stopped at a single directory level
while match() on the same Glob object (which consults pattern_slice(),
already stripping the separator) accepted arbitrarily deep paths.

Detect trailing_sep first and classify on the same slice pattern_slice()
returns. "**/" now recurses like "**" while still filtering out files
via the existing trailing_sep check in match_pattern_file().
Widen the trailing-separator fix to the two sibling sites that still
admitted files:

- match_pattern_file's `**/X` peek-ahead checked pattern.trailing_sep
  (the globstar's) but not next_pattern.trailing_sep, so "**/foo/"
  and "**/*/" yielded regular files.
- The Literal-tail statat() fast path accepted S::ISREG without
  consulting trailing_sep, so "foo/" and "sub/foo/" yielded a file.
@robobun
robobun force-pushed the farm/06067573/glob-trailing-slash-globstar branch from 95401e4 to ceadce4 Compare July 22, 2026 00:54
@robobun

robobun commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto main (85ddc95). The only conflict was with #35002's dead-code removal of the is_ascii block in make_component, which overlapped the block this PR relocates; resolved by dropping both (trailing_sep detection is now at the top of the function and the is_ascii field no longer exists). All 263 glob-related tests pass on the rebased branch.

Comment thread src/glob/GlobWalker.rs
@robobun

robobun commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator Author

Do not merge this PR yet. It changes a form that works on main.

A pattern whose last segment is ./ or ../ lists one directory on main. With this PR it lists nothing. I ran both on Linux x64, with scanSync({ cwd, onlyFiles: false }) in a tree dir/sub:

Pattern main (bd599f5) main with this PR
./ ["."] []
../ [".."] []
dir/./ ["dir/."] []
dir/../ ["dir/.."] []
dir/sub/../ ["dir/sub/.."] []

The cause is in make_component. The PR compares the component with . and .. after it removes the trailing separator. So a final ./ becomes SyntaxHint::Dot and a final ../ becomes SyntaxHint::DotBack. The walker yields nothing for a pattern that ends in one of those. On main both are SyntaxHint::Literal, and the statat() fast path yields the directory.

The PR body and my earlier review reply called the ../ change a consistency gain. That was wrong. The change removes a result that main returns, and the fix for **/ does not need it.

I will change the PR so that these five patterns keep the result they have on main, and I will add them as tests.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants