Skip to content

feat(ext/node): add experimental node:vfs polyfill#34644

Closed
divybot wants to merge 9 commits into
mainfrom
orch/divybot-371
Closed

feat(ext/node): add experimental node:vfs polyfill#34644
divybot wants to merge 9 commits into
mainfrom
orch/divybot-371

Conversation

@divybot

@divybot divybot commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds an experimental node:vfs polyfill to Deno, providing an in-memory virtual filesystem with an fs-compatible API.

The implementation is ported from @platformatic/vfs (the package extracted from nodejs/node#61478, which itself is currently open and experimental).

What's included

  • VirtualFileSystem with sync, callback, and promises APIs.
  • MemoryProvider (the default storage backend) and VirtualProvider base class for custom providers.
  • File-descriptor table for openSync/readSync/closeSync/fstatSync.
  • createReadStream (Readable) lazily wired to node:stream.
  • Mount / unmount state tracking and process.emit("vfs-mount" / -unmount) events.
  • VirtualStats + VirtualDirent fs-compatible shapes.
  • Symlink + lstat/realpath resolution with loop detection.
  • Optional virtual cwd ({ virtualCwd: true } patches process.cwd/process.chdir).
  • TypeScript definitions in cli/tsc/dts/node/vfs.d.cts.

What's intentionally left out

Node's mount() machinery patches Node-internal require(), Module._resolveFilename, and core fs functions (readFileSync, statSync, existsSync, readdirSync, realpathSync, watch, ...) so that the rest of the process transparently sees the virtual files. That half doesn't translate to Deno's loaders and is intentionally not implemented in this polyfill - the VirtualFileSystem instance itself is the namespace through which callers interact with the virtual files. Mount/unmount state is still tracked, so user code that branches on vfs.mounted or listens for the vfs-mount/vfs-unmount events continues to work.

Example

import { create } from "node:vfs";

const vfs = create();
vfs.mkdirSync("/app", { recursive: true });
vfs.writeFileSync("/app/file.txt", "hello");
vfs.readFileSync("/app/file.txt", "utf8"); // 'hello'

// Mount, observe the lifecycle:
vfs.mount("/app");
vfs.shouldHandle("/app/file.txt"); // true

// Promises API:
await vfs.promises.readFile("/app/file.txt", "utf8");

Test plan

  • cargo build --bin deno (clean build)
  • cargo build -p deno_node (cargo check)
  • tools/lint.js --js
  • deno fmt --check clean for changed files
  • target/debug/deno test --no-check --config import_map.json tests/unit_node/vfs_test.ts - 26 passed, 0 failed

The new spec coverage exercises:

  • writeFileSync/readFileSync round-trip (Buffer + string encoding paths)
  • existsSync (present + missing)
  • statSync (size, isFile/isDirectory)
  • mkdirSync recursive + readdirSync (with and without withFileTypes)
  • unlinkSync, renameSync, copyFileSync
  • appendFileSync (multi-step append)
  • ENOENT / EEXIST / EROFS error codes
  • readonly provider via MemoryProvider.setReadOnly()
  • symlinkSync / readlinkSync / lstatSync + symlink-following statSync
  • File descriptor open/read/fstat/close round-trip
  • mount + shouldHandle + unmount lifecycle, double-mount throws
  • Callback API readFile/writeFile
  • Promises API readFile/writeFile/mkdir/readdir/stat + missing-file rejection
  • Custom provider extending VirtualProvider
  • createReadStream consumed via for await
  • virtualCwd: true chdir/cwd round-trip + cwd() throws when disabled
  • internalModuleStat return values for dirs, files, and missing

Closes denoland/divybot#371

Ports the in-memory Virtual File System from nodejs/node#61478 (and its
extracted npm package @platformatic/vfs) as a Deno polyfill, available
via `import "node:vfs"`.

Includes:
- VirtualFileSystem with sync, callback, and promises APIs.
- MemoryProvider (the default storage backend) and VirtualProvider base
  class for custom providers.
- File-descriptor table for openSync/readSync/closeSync/fstatSync.
- createReadStream (Readable) lazily wired to node:stream.
- Mount/unmount state tracking and process emit("vfs-mount" / -unmount).
- VirtualStats + VirtualDirent fs-compatible shapes.

Node's mount() machinery patches require() and core fs functions; that
half doesn't translate to Deno's loaders and is intentionally left out.
The VFS instance is the namespace through which callers interact with
the virtual files. cwd hooking is provided when `virtualCwd: true`.

Co-Authored-By: Divy Srivastava <me@littledivy.com>
@divybot divybot changed the title [denoland/deno#32783] Support node:vfs feat(ext/node): add experimental node:vfs polyfill Jun 1, 2026
divybot and others added 3 commits June 1, 2026 09:24
`functionExpression.flatIife` in .dprint.json keeps the IIFE body
un-indented. The previous commit went through `deno fmt`, which added
2-space indentation to the IIFE body and tripped tools/format.js in CI.

Co-Authored-By: Divy Srivastava <me@littledivy.com>
…ns list

- Remove `override` keywords on members of `CustomProvider` (extends
  `VirtualProvider as any`); TS4113 fires because the base resolves to
  `any` and override-checking can't see the parent method.
- Cast the CustomProvider instance at the `create()` call site instead.
- Extend the expected node-specifier list in
  `tests/integration/lsp_tests.rs::lsp_completions_node_specifier` with
  `node:vfs` so the LSP completions test matches the new builtin.

Co-Authored-By: Divy Srivastava <me@littledivy.com>
@littledivy

Copy link
Copy Markdown
Member

This is the PR that merged in Node. Base your implementation on this nodejs/node#63115

@bartlomieju bartlomieju left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has this even landed in Node yet?

@divybot

divybot commented Jun 2, 2026

Copy link
Copy Markdown
Contributor Author

Yes — landed upstream as nodejs/node#63115, merged 2026-05-23 (commit c9562ddb82dde83eb6e9c62aa8055b345a46b575) and shipped behind --experimental-vfs. The original PR description here pointed at the broader nodejs/node#61478, which is still open and tracks the mount() / require-hook integration as a follow-up.

@littledivy flagged the same thing yesterday, and I've just force-pushed a rewrite to base the implementation on the merged sub-PR. New surface matches upstream: VirtualFileSystem, VirtualProvider, MemoryProvider, RealFSProvider, VirtualFileHandle / MemoryFileHandle, VirtualDir, the polling VFSWatcher / VFSStatWatcher / VFSWatchAsyncIterable, plus the full sync / callback / promises method set documented in doc/api/vfs.md. The Deno-specific extensions from the previous draft (mount(), virtualCwd, internalModuleStat, vfs-mount/-unmount process events) have been dropped to keep parity. Stats are real fs.Stats instances now.

divybot and others added 5 commits June 2, 2026 16:46
The original draft was ported from @platformatic/vfs (extracted from the
still-open nodejs/node#61478). Node has since merged the experimental
subsystem in nodejs/node#63115, with a slightly different public surface.
Rebase the polyfill on the merged source so the API matches what landed
in Node behind --experimental-vfs:

- Stats objects are now real fs.Stats / BigIntStats instances (no more
  custom VirtualStats shape).
- Adds RealFSProvider, VirtualDir, plus the polling watcher classes
  (VFSWatcher / VFSStatWatcher / VFSWatchAsyncIterable).
- Adds rmSync/rm, truncateSync, linkSync, chmodSync, chownSync,
  utimesSync, lutimesSync, mkdtempSync, opendirSync, openAsBlob, the
  matching callback wrappers, and the matching promises API.
- Drops the Deno-specific extensions from the previous draft
  (mount/unmount/mounted/shouldHandle, virtualCwd/chdir, vfs-mount and
  vfs-unmount process events, internalModuleStat); they have no
  counterpart in the merged Node API and the module-loader integration
  is intentionally left for a follow-up upstream.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop unused Symbol import, replace const self = this with a closure
helper, drop async from the RealFSProvider test (no awaits inside).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
internal/fs/utils.mjs is registered as a lazy-loaded ESM module, so it
must be reached via core.createLazyLoader rather than core.loadExtScript
(the latter is only valid for IIFE-style polyfills). Without this, every
debug build that exercised node:vfs threw at module init time:

  Error: Script "ext:deno_node/internal/fs/utils.mjs" cannot be
  lazy-loaded as it was not included in the binary.

That bypassed all unit_node / specs / node_compat suites on the runners
that didn't bundle a fresh release snapshot. Wrap Stats / BigIntStats /
Dirent behind a memoized loader and call into it from buildStats,
createZeroStats, and readdirSync.

Also open the underlying fd synchronously in VirtualReadStream so the
first _read() always finds it, mirroring fs.createReadStream's contract
and avoiding a "Promise resolution is still pending" abort under deno
test.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
unit_node CI runs with type-checking on and tripped on:
- assertInstanceOf(st, Stats): node:fs Stats has a private constructor.
  Walk the prototype chain instead.
- st.mode & 0o777: st.mode is typed as number | bigint, narrow with a
  cast since the non-bigint statSync path returns a number.
- CustomProvider in the custom-provider test had a get readonly()
  accessor where the base class declares a readonly property, and
  missing override modifiers on statSync/readFileSync/writeFileSync.
  Drop the readonly accessor (false is the inherited default) and tag
  the overrides.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous fix worked around the private-constructor type error by
calling Stats.prototype.isPrototypeOf, which trips the
no-prototype-builtins lint rule. Cast Stats through unknown for the
instanceof check instead.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@littledivy

Copy link
Copy Markdown
Member

It's experimental. Let's wait for stable. Closing to free queue

@littledivy littledivy closed this Jun 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants