Skip to content

shell: hand mkdir/touch/cp operands to the kernel as written - #41903

Open
robobun wants to merge 2 commits into
mainfrom
robobun/b8dc66ab/shell-no-lexical-normalize
Open

robobun wants to merge 2 commits into
mainfrom
robobun/b8dc66ab/shell-no-lexical-normalize

Conversation

@robobun

@robobun robobun commented Sep 8, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • The mkdir, touch and cp shell builtins folded . and .. out of each operand, path.join style, before any syscall. mkdir -p d/link/../nd created d/nd where coreutils (and ls, cat, mv, redirects in the same shell) see other/nd, and touch d/missing/../t created d/t instead of failing with ENOENT.
  • Cause: mkdir.rs, touch.rs and cp.rs joined the operand onto the shell cwd with resolve_path::join_z*, which normalizes like Node's path.join.

Fix

  • New shell_join_cwd (interpreter.rs): a relative operand gets the shell cwd prefixed and nothing else changes. Windows keeps the normalizing join, because Win32 resolves .. textually itself.
  • cp refused to copy a file onto itself by comparing the two joined strings. Two spellings of one path no longer compare equal, so the check now compares device and inode once the target has its final dir/basename form, as BSD and GNU cp do.
  • Verified: test/js/bun/shell/commands/{mkdir,touch,cp}.test.ts, new cases fail on 1.4.3 and pass here. Also the ls, mv, rm and bunshell suites.

Background

  • The shell keeps its own cwd as a string and an fd. mkdir, touch and cp sit on path-only APIs (node:fs mkdir and cp, utimens), so they spell the operand from the cwd string.
  • To the kernel, a/link/.. is the parent of the link target and missing/.. does not exist. Only a textual resolver (Node's path, Win32) reads them as a and ..
  • cp is a builtin by default only on Windows; on POSIX the new cp tests enable it with BUN_ENABLE_EXPERIMENTAL_SHELL_BUILTINS=1.
Notes
  • Scope. The report this answers (forwarded by @Jarred-Sumner) also covers rm: rm -r listed a directory through the kernel path but unlinked each entry under the lexically joined parent, so rm -rf d/link/.. deleted same-named files in d/. An earlier revision of this PR fixed that too (file entries unlinked relative to the listed directory's fd, plus a ./.. operand refusal). That part is dropped here because shell(rm): resolve every entry relative to the directory fd the walk holds #41842 reworks the whole rm walk to resolve every entry, directories included, relative to a held parent fd, which fixes the same bug and the directory-level re-resolution race, and the two conflicted throughout rm.rs. The rm regression cases from that revision are posted on shell(rm): resolve every entry relative to the directory fd the walk holds #41842. The POSIX ./.. operand refusal (rm -rf . empties the cwd today) is shell(rm): refuse '.' and '..' operands instead of emptying the directory #34906's subject and is not included here either.
  • Side effects: a relative mkdir/touch operand longer than PATH_MAX now fails with ENAMETOOLONG instead of being normalized short, as coreutils does (the existing long-operand tests change accordingly); mkdir '' and touch '' fail with ENOENT instead of acting on the cwd (shell: fail an empty operand with ENOENT instead of acting on the cwd #38002 covers empty operands more broadly, including the Windows fd-relative builtins).
  • mkdir -pv d/missing/../x creates d/missing and then d/x, printing both, exactly as GNU does; mkdir d/file/../x is ENOTDIR.
  • The same-file check also covers hard links, a symlink to the file as either operand (its identity is the file it points at), and a directory reached through a symlink. On Windows there is no inode at hand, so the check stays a string compare and shell_join_cwd keeps normalizing both relative and absolute operands there. The macOS copy path unlinks the destination before clonefile for files over 128 KB, so a same-file copy that slipped past the builtin's check would not be harmless there; the new cp test includes a 200 KB file copied onto ./itself and through an absolute spelling.
  • An earlier revision (with the rm part) also passed its tests on Windows x64; cargo check for x86_64-pc-windows-msvc and aarch64-apple-darwin passes on this one.
  • GNU control, for reference: mkdir -p d/link/../nd creates other/nd; touch d/nodir/../t is ENOENT; cp a ./a is refused ("are the same file"); touch '' and mkdir '' fail.

no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/shell/commands/touch.test.ts, test/js/bun/shell/commands/mkdir.test.ts, test/js/bun/shell/commands/cp.test.ts

@coderabbitai

coderabbitai Bot commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 66f12dcd-7688-48a9-8632-172a98bebc7c

📥 Commits

Reviewing files that changed from the base of the PR and between 7084d52 and d936b02.

📒 Files selected for processing (6)
  • src/runtime/shell/builtin/cp.rs
  • src/runtime/shell/builtin/mkdir.rs
  • src/runtime/shell/builtin/rm.rs
  • src/runtime/shell/builtin/touch.rs
  • src/runtime/shell/interpreter.rs
  • test/js/bun/shell/commands/rm.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 1 remains after this review.


Walkthrough

Changes

The shell path builtins now use shared cwd-relative joining. POSIX operands preserve kernel handling of . and ..; Windows operands use textual normalization. rm uses descriptor-relative deletion and rejects operands naming . or ...

Changes

Shell path semantics

Layer / File(s) Summary
CWD-relative paths for file builtins
src/runtime/shell/interpreter.rs, src/runtime/shell/builtin/{cp,mkdir,touch}.rs, src/sys/lib.rs, test/js/bun/shell/commands/{cp,mkdir,touch}.test.ts
Adds shell_join_cwd and applies it to cp, mkdir, and touch. Windows directory names retain NUL-terminated UTF-8 data. Tests cover symlink traversal, long operands, and platform-specific behavior.
Descriptor-relative recursive removal
src/runtime/shell/builtin/rm.rs, test/js/bun/shell/commands/rm.test.ts
Introduces EntryRef for directory-descriptor-relative unlinking, preserves kernel path resolution during recursion, and rejects operands ending in . or ... Tests cover symlink races, PATH_MAX operands, protected paths, and recursive removal.

Merge Risk: ⚪ Minimal · up to d936b

The shell builtins now preserve intended path-resolution semantics, with descriptor-relative recursive removal and protected rm operands covered by the supplied tests. No current merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main behavior change for the shell builtins. It is concise and specific, although it does not mention the related rm changes.
Description check ✅ Passed The description explains the problem, cause, fix, scope, platform behavior, side effects, and verification coverage. It does not use the exact template headings, but it provides the required informati…

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the claude label Sep 8, 2026
@robobun

robobun commented Sep 8, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 11:07 AM PT - Sep 8th, 2026

✅ @robobun, your commit 213d4ca8b7ea37d6205ebc469b027e25055bc198 passed in Build #113058! 🎉


🧪   To try this PR locally:

bunx bun-pr 41903

That installs a local version of the PR into your bun-41903 executable, so you can run:

bun-41903 --bun

@robobun

robobun commented Sep 8, 2026 •

Copy link
Copy Markdown
Collaborator Author

Rescoped. This PR now covers mkdir, touch and cp only (operands spelled from the shell cwd without normalization, and cp's same-file check done by device and inode). The rm half of the report, fd-relative unlinking of listed entries, goes through #41842, which reworks the whole walk that way; the regression cases from the earlier revision here are posted there. The ./.. operand refusal is #34906's subject.

Reproduced on 1.4.3 (Linux), in a temp dir with d/c.txt, other/c.txt, other/sub/ and d/link -> ../other/sub:

$ mkdir -p d/link/../nd   -> d/nd         (coreutils: other/nd)
$ touch d/nodir/../tt     rc=0 -> d/tt    (coreutils: ENOENT, rc 1)
$ touch ''                rc=0, bumps the cwd's mtime
$ cp a ./a                refused only because both strings normalized equal

With this branch: mkdir -p d/link/../nd creates other/nd, touch d/nodir/../tt and touch '' fail with ENOENT, cp d/c.txt ./d/c.txt, cp d/c.txt d, a hard link and a path through a symlinked directory are all refused as the same file. The new cases in test/js/bun/shell/commands/{mkdir,touch,cp}.test.ts fail on 1.4.3 and pass here.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Comment thread test/js/bun/shell/commands/rm.test.ts Outdated
Comment thread src/runtime/shell/builtin/cp.rs Outdated
Comment thread src/runtime/shell/builtin/cp.rs Outdated
Comment thread src/runtime/shell/builtin/mkdir.rs Outdated
Comment thread src/runtime/shell/builtin/rm.rs Outdated
Comment thread src/runtime/shell/builtin/rm.rs Outdated
Comment thread src/runtime/shell/builtin/rm.rs Outdated
Comment thread src/runtime/shell/builtin/rm.rs Outdated
Comment thread src/runtime/shell/builtin/rm.rs Outdated
Comment thread src/runtime/shell/builtin/rm.rs Outdated
Comment thread src/runtime/shell/builtin/rm.rs Outdated
Comment thread src/runtime/shell/builtin/touch.rs Outdated
Comment thread src/runtime/shell/interpreter.rs Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Comment thread src/runtime/shell/builtin/rm.rs Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

The mkdir, touch and cp builtins joined each operand onto the shell cwd
with a normalizing join, so `.` and `..` components were folded out of
the string before any syscall saw it. `mkdir -p d/link/../nd` created
`d/nd` instead of the directory next to the link target, and
`touch d/missing/../t` succeeded instead of failing with ENOENT.

The new shell_join_cwd prefixes a relative operand with the shell cwd and
changes nothing else (Windows keeps the normalizing join, since Win32
resolves `..` textually itself). cp's refusal to copy a file onto itself
compared the two joined strings, which no longer match for two spellings
of one path, so it now compares device and inode after the target has
its final form.
@robobun
robobun force-pushed the robobun/b8dc66ab/shell-no-lexical-normalize branch from d936b02 to b1fb0da Compare September 8, 2026 15:52
@robobun robobun changed the title shell: hand rm/mkdir/touch/cp operands to the kernel as written shell: hand mkdir/touch/cp operands to the kernel as written Sep 8, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Comment thread src/runtime/shell/builtin/cp.rs
Comment thread src/runtime/shell/builtin/cp.rs
…le check; normalize absolute operands on Windows again

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — 213d4ca addresses both points from the last pass: shell_join_cwd now routes absolute operands through the normalizing join on Windows so the byte-equality same_file check holds, and classify() follows a symlink operand via stat (falling back to the link's own id when dangling) so cp file link-to-file and the reverse are refused; the new ontoSymlink/fromSymlink test cases cover it.

What was reviewed:

  • shell_join_cwd: empty-operand early return, POSIX plain-separator join vs Windows normalizing join_z_spill, absolute-path handling per platform.
  • cp same-file: (st_dev, st_ino) comparison after target-dir join, re-classify on the joined dir/basename path, error message shows the user's operands (not the cwd-prefixed form).
  • Confirmed the dropped rm.rs/rm.test.ts changes are no longer in the diff, so the earlier setDefaultTimeout and ./.. diagnostic threads are moot for this PR.
Extended reasoning...

Overview

This PR replaces the textual resolve_path::join_z* normalization of mkdir/touch/cp shell-builtin operands with a shared shell_join_cwd helper that, on POSIX, only prefixes the shell cwd and hands the path to the kernel byte-for-byte — so link/../x resolves through the filesystem as every other program sees it. Windows keeps the normalizing join (Win32 resolves .. textually and NT calls reject dot components). cp also gains a real same-file check: classify() returns (st_dev, st_ino) (following a symlink operand to its target) and same_file() compares those on POSIX, with normalized-path byte equality on Windows. Since the previous review round, commit 213d4ca addressed both open inline comments (Windows absolute-operand normalization; symlink-aware identity), added test coverage for the symlink-as-operand cases, and the rm portion was dropped from the PR entirely in favor of #41842.

Security risks

Path resolution through symlinks is security-adjacent, but this change moves the builtins toward kernel semantics (the same view ls, cat, redirects, and coreutils have) rather than away from them, eliminating a class of "operates on a different file than the user typed" surprises. The shell runs with the invoking user's privileges and the cp builtin is behind BUN_ENABLE_EXPERIMENTAL_SHELL_BUILTINS on POSIX. No new attack surface is introduced; the same-file refusal is a safety guard that now covers strictly more cases (hard links, symlinks, alternate spellings) than the base branch's string equality.

Level of scrutiny

Moderate. The change is focused (one shared helper, three call sites, one identity check), the reference behavior is well-specified (GNU/BSD cp, coreutils mkdir/touch), and the test matrix is thorough — symlink .. traversal for all three builtins, an eight-way same-file refusal matrix including the >128 KB macOS unlink-first path, and updated ENAMETOOLONG expectations for the ./-padded operands. The cross-platform #[cfg] split is the main risk area; bun run rust:check-all is warranted but the Windows branch is a small, mechanical variation on the base-branch code (same join_z_spill, now also applied to absolute operands, restoring base-branch behavior there).

Other factors

This is the sixth review pass. All prior inline threads from this reviewer are either addressed by 213d4ca (the two on cp.rs) or moot because the rm changes were dropped from the diff. No human CHANGES_REQUESTED reviews are outstanding, no CODEOWNERS cover the touched paths, and the bug hunt exited on dry_streak with no findings and no ruled-out candidates. The PR description explicitly defers Windows verification to CI (the new POSIX-only tests are test.skipIf(isWindows) with stated reasons); the Windows code path is unchanged in behavior from base except for re-normalizing absolute operands, which restores what base already did.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants