Skip to content

node:fs: keep Buffer paths as bytes through cp, cpSync, and promises.cp - #41112

Open
robobun wants to merge 2 commits into
mainfrom
robobun/9b38468b/fs-cp-buffer-paths
Open

robobun wants to merge 2 commits into
mainfrom
robobun/9b38468b/fs-cp-buffer-paths

Conversation

@robobun

@robobun robobun commented Sep 1, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Fix

  • A Buffer path stays bytes to the syscalls. The checks and the walker run their node:path arithmetic on a latin1 view of the bytes, then turn the result back into a Buffer.
  • Entry names and link targets under a Buffer path are read with { encoding: "buffer" }, so names that are not UTF-8 copy byte for byte. filter and error objects get the Buffers they were given. String paths are unchanged.
  • Verified: test/js/node/fs/cp.test.ts (7 new tests, all fail on 1.4.1, one is Linux only). Also cp-symlink-target.test.ts and the 77 vendored node test-fs-cp-* tests.
  • Self-reviewed: 3 changes asked, all done (push the bytes design, describe it here, cite readdir with withFileTypes and encoding "buffer" returns undefined for Dirent.name #27914 at the opendir fork).

Background

  • fs.cpSync validates the arguments, runs node's checks in JS, then calls the native binding or the ported walker (internal/fs/cp-sync.ts, async in internal/fs/cp.ts).
  • The fs functions the walker calls and the native binding already pass a buffer's bytes to the syscall unchanged. Only the path arithmetic needed a string.
  • latin1 is lossless: latin1Slice maps each byte to one code unit and Buffer.from(view, "latin1") maps it back, and node:path inspects only ASCII.
Notes
  • resolve() falls back to process.cwd(), a UTF-16 string, so the helpers pass a latin1 view of the cwd in explicitly when they resolve a Buffer path.
  • The first version of this PR decoded the Buffer to a UTF-8 string once after validation. Jarred asked to keep the bytes instead. This version does.
  • Node v26.3.0: cpSync accepts a Buffer path for a file or for a directory without filter (its checks and directory copy are C++). With filter, and in fs.cp and fs.promises.cp, node throws ERR_INVALID_ARG_TYPE from path.join or path.dirname, because its JS walker has the same shape as the one ported here. Node documents src and dest as string | URL. This PR accepts Buffers in all three, as bun 1.3.14 did, and in the walker too.
  • readdirSync(dir, { withFileTypes: true, encoding: "buffer" }) and opendir(dir, { encoding: "buffer" }) return entries without usable names in bun today (fs: copy directory entries with non-UTF-8 names byte-exact in cp/cpSync #36064 covers the first). The walker only needs names, so it lists a Buffer directory with readdir(dir, { encoding: "buffer" }). The macOS clonefile pre-scan still uses withFileTypes, so a directory name that is not UTF-8 makes the scan bail to the walker, which is the conservative outcome.
  • fs: copy directory entries with non-UTF-8 names byte-exact in cp/cpSync #36064 (open) makes entry names that are not UTF-8 byte-exact for string paths, with the same latin1 view approach. The two changes touch the same functions and will need a rebase against each other.
  • The self-review's probes found two things outside this PR. A symlink copied over an existing regular file fails with EEXIST, for Buffer and string paths alike, and node does the same (its onLink comments that fs throws anyway). And fs.openAsBlob(Buffer) trips a JSC debug assertion (JSCell::classInfo during MutatorState::Sweeping) under GC pressure: the Blob store drops its pinned buffer inside the finalizer, and JSC__JSValue__unpinArrayBuffer inspects the cell there. String paths do not. Neither involves cp.
  • Before the port, fs.cpSync(src, dest) with no options went straight to the native binding (if (!options) return fs.cpSync(src, dest)), which is why 1.3.14 worked.
  • Repro:
const fs = require("node:fs"), path = require("node:path"), os = require("node:os");
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "cp-"));
const src = path.join(tmp, "a.txt"); fs.writeFileSync(src, "hello");
fs.cpSync(Buffer.from(src), path.join(tmp, "b.txt"));
console.log(fs.readFileSync(path.join(tmp, "b.txt"), "utf8"));

bun 1.4.1: TypeError: The "path" property must be of type string, got object at checkParentPathsSync (internal:fs/cp-sync:147:34). bun 1.3.14 and node v26.3.0: hello.


no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/node/fs/cp.test.ts

…es.cp

The node cp port (#31830) runs node:path checks on the validated src and
dest before the native copy. getValidatedFsPath passes a Buffer through
unchanged, so path.dirname threw ERR_INVALID_ARG_TYPE. bun 1.3.14 took
Buffer paths in all three functions.

Decode the bytes to a string once, after validation, in a helper shared
by the three entry points.
@coderabbitai

coderabbitai Bot commented Sep 1, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 7d08db34-6440-40fb-b987-beb256042c64

📥 Commits

Reviewing files that changed from the base of the PR and between 747e8b4 and 234f1f4.

📒 Files selected for processing (3)
  • src/js/internal/fs/cp-sync.ts
  • src/js/internal/fs/cp.ts
  • test/js/node/fs/cp.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 0 remain after this review.


Walkthrough

fs.cp and fs.cpSync now preserve Buffer and Uint8Array path bytes during parent resolution, directory traversal, filtering, and symlink handling. Tests cover recursive copies, callbacks, filters, and invalid UTF-8 filenames.

Changes

Buffer-aware filesystem copying

Layer / File(s) Summary
Buffer path and symlink helpers
src/js/internal/fs/cp-sync.ts, src/js/internal/fs/cp.ts
Shared helpers detect byte paths, select encodings, join directory entries, resolve parents, and normalize symlink targets.
Copy traversal and parent validation
src/js/internal/fs/cp-sync.ts, src/js/internal/fs/cp.ts
Synchronous and asynchronous copy paths preserve Buffer representations during parent checks, directory reads, recursive copying, and symlink processing.
Buffer path and byte preservation tests
test/js/node/fs/cp.test.ts
Tests cover Buffer and Uint8Array paths, filters, callback copying, recursive traversal, invalid filename bytes, and relative symlink targets.

Suggested reviewers: cirospaciari

Merge Risk: ⚪ Minimal · up to 234f1

The change preserves Buffer paths through copy operations while keeping string-path behavior unchanged, and no actionable merge-blocking risk remains beyond normal checks and review.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely identifies the main change: preserving Buffer paths across fs.cp, cpSync, and promises.cp.
Description check ✅ Passed The description provides a detailed problem statement, implementation summary, verification results, background, and testing context. It does not use the exact template headings, but it covers the req…
Full details: Description check

Explanation

The description provides a detailed problem statement, implementation summary, verification results, background, and testing context. It does not use the exact template headings, but it covers the required information for what the PR does and how the code was verified.


Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the claude label Sep 1, 2026
@robobun

robobun commented Sep 1, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: reworked per review, ready for another look.

Reproduced on bun 1.4.1 and on a debug build of main with fs.cpSync(Buffer.from(src), dest): TypeError [ERR_INVALID_ARG_TYPE]: The "path" property must be of type string, got object, thrown from checkParentPathsSync in internal/fs/cp-sync. fs.cp, fs.promises.cp, and a plain Uint8Array path fail the same way. Bun 1.3.14 copies the file in all three cases, so this is a regression from the node cp port in #31830.

The fix keeps a Buffer path as bytes through the node checks, the walker, and the native copy. The path arithmetic runs on a latin1 view of the bytes and goes back to bytes. The 7 new tests in test/js/node/fs/cp.test.ts fail on 1.4.1 and pass with this branch. The 77 vendored test-fs-cp-* node tests and cp-symlink-target.test.ts also pass.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — small, well-scoped regression fix restoring Buffer/Uint8Array path support in the cp family.

What was reviewed:

  • getValidatedCpPath correctly wraps getValidatedFsPath and decodes non-string results via BufferPrototypeToString.$call(p), matching the file's existing primordial pattern and the .$call convention.
  • All three entry points (cpSync, callback cp, promises.cp) are switched consistently; the dropped require("internal/validators") at each site is now dead since the helper moved into cp-sync.
  • Tests cover Buffer src, Buffer dest, plain Uint8Array, recursive directories, filter-receives-strings, and the callback form; they're in the existing cp.test.ts using tempDir/await using per harness conventions.
Extended reasoning...

Overview

This PR fixes a regression in fs.cp, fs.cpSync, and fs.promises.cp where Buffer/Uint8Array path arguments stopped working after the Node cp port landed. The fix adds a single helper getValidatedCpPath in src/js/internal/fs/cp-sync.ts that runs the existing getValidatedFsPath validator (which passes Uint8Array through unchanged) and then decodes any non-string result to a UTF-8 string so the JS-side node:path operations (dirname, resolve, join) can consume it. The three dispatchers in src/js/node/fs.ts and src/js/node/fs.promises.ts are switched to the new helper, dropping their now-redundant separate require("internal/validators") import. Five new tests are added to the existing test/js/node/fs/cp.test.ts.

Security risks

None. The change decodes user-supplied path bytes to a string before handing them to the same validation and copy machinery that already handles string paths. Null-byte rejection happens in getValidatedFsPath before the decode, on the raw bytes. No new privilege boundaries, no new parsing of untrusted structured data.

Level of scrutiny

Low-to-moderate. This is a ~10-line runtime change plus test additions. The helper follows the file's established primordial-capture pattern exactly (const BufferPrototypeToString = Buffer.prototype.toString; alongside the existing ArrayPrototypeEvery, StringPrototypeSplit) and uses .$call per the builtins tamper-proofing convention. Buffer.prototype.toString called on a plain Uint8Array works correctly since Buffer extends Uint8Array. The comment on the helper is concise and explains the "why" (node's cpSync C++ takes raw bytes; the JS walker uses node:path which needs strings). The PR description transparently notes the UTF-8-decode limitation for non-UTF-8 byte paths as an explicit scope decision, which is reasonable — Bun 1.3.14 passed bytes straight to native, but the ported walker already decodes child names as UTF-8, so this doesn't newly break anything.

Other factors

Test coverage is thorough for the size of the fix: the parameterized impls loop exercises both cpSync and promises.cp with Buffer src, Buffer dest, plain Uint8Array, and a recursive directory copy; a separate test covers the callback form; the filter test asserts the documented Node contract that filter receives string paths and sorts its captured pairs to avoid ordering flakiness. Tests use tempDir with await using, live in the existing module test file, and would fail on the pre-fix build (the path.dirname call would throw ERR_INVALID_ARG_TYPE). No CODEOWNERS cover the touched paths. Exit reason was dry_streak with no candidate findings.

@robobun

robobun commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator Author

The review requests no changes, so the branch stays as pushed (15603cd). Buildkite build #109045 is still running. The red claude-find-issues check is the duplicate-finder action failing on its own startup, not a test in this PR.

@Jarred-Sumner Jarred-Sumner left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep it as a buffer, don't decode UTF-8

@robobun

robobun commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator Author

Understood. I will rework this so the Buffer stays a Buffer: the node checks and the walker will run their path arithmetic on a latin1 view of the bytes (one byte per code unit, so it round-trips) and pass the bytes themselves to every syscall. I will update the PR when it is pushed.

Comment thread src/js/internal/fs/cp-sync.ts Outdated
@robobun

robobun commented Sep 2, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 8:42 PM PT - Sep 1st, 2026

✅ @robobun, your commit 234f1f4aaaf8958d56c2056614dea1ac6da8fe5d passed in Build #109197! 🎉


🧪   To try this PR locally:

bunx bun-pr 41112

That installs a local version of the PR into your bun-41112 executable, so you can run:

bun-41112 --bun

Comment thread src/js/internal/fs/cp-sync.ts Outdated
Comment thread src/js/internal/fs/cp-sync.ts Outdated
Comment thread src/js/internal/fs/cp.ts Outdated
@robobun
robobun force-pushed the robobun/9b38468b/fs-cp-buffer-paths branch from fe3a4c8 to 8aa45e6 Compare September 2, 2026 03:21
Comment thread src/js/internal/fs/cp-sync.ts Outdated
Comment thread src/js/internal/fs/cp-sync.ts Outdated
The node cp port (#31830) runs node:path checks and a JS walker on the
validated src and dest before the native copy. getValidatedFsPath passes
a Buffer through unchanged, so path.dirname threw ERR_INVALID_ARG_TYPE.
bun 1.3.14 took Buffer paths in all three functions.

A Buffer path now stays bytes all the way to the syscall. The checks and
the walker run their dirname/join/resolve on a latin1 view of the bytes
(one byte per code unit, and node:path inspects only ASCII) and turn the
result back into bytes. Entry names and link targets under a Buffer path
are read as bytes too. The filter and the error objects get the Buffers
they were given.
@robobun
robobun force-pushed the robobun/9b38468b/fs-cp-buffer-paths branch from 8aa45e6 to 234f1f4 Compare September 2, 2026 03:23
@robobun robobun changed the title node:fs: accept Buffer and Uint8Array paths in cp, cpSync, and promises.cp node:fs: keep Buffer paths as bytes through cp, cpSync, and promises.cp Sep 2, 2026
@robobun

robobun commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator Author

Reworked as asked, pushed as 234f1f4: the Buffer stays a Buffer.

  • No decode anywhere. fs.ts and fs.promises.ts are back to main. The validated Uint8Array reaches every syscall (stat, copyFile, mkdir, readdir, readlink, symlink, chmod, utimes, unlink) and the native cp binding as bytes.
  • The node checks and the walker (internal/fs/cp-sync.ts, internal/fs/cp.ts) run their dirname/join/resolve/isAbsolute on a latin1 view of the bytes (one byte per code unit) and turn the result back into a Buffer. node:path only inspects ASCII, so the view round-trips. The cwd is viewed the same way, because resolve() would otherwise mix in a UTF-16 string.
  • Entry names under a Buffer directory come from readdir(dir, { encoding: "buffer" }) and link targets from readlink(link, { encoding: "buffer" }), so a tree whose names are not UTF-8 copies byte for byte. The filter callback and the error objects get the Buffers they were given.
  • String paths take the same code as before.

Tests: 7 new cases in test/js/node/fs/cp.test.ts, all fail on 1.4.1. One (Linux only) copies a tree named from\xff with a caf\xe9.txt entry and a relative symlink, and checks the destination names and the copied link target byte for byte. The 77 vendored test-fs-cp-* node tests and cp-symlink-target.test.ts still pass. Probes of every option (preserveTimestamps, dereference, verbatimSymlinks, errorOnExist, force, mode), mixed string and Buffer inputs, relative Buffer paths, and the ERR_FS_CP_* error paths pass with Buffers.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants