fs,doc,test: open reserved characters under win32 #13875

XadillaX · 2017-06-22T16:49:27Z

Explain the behavior of fs.open() under win32 that file path contains
some characters and add some test cases for them.

< (less than)
> (greater than)
: (colon)
" (double quote)
/ (forward slash)
\ (backslash)
| (vertical bar or pipe)
? (question mark)
* (asterisk)

Refs: #13868
Refs: https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx
Refs: https://msdn.microsoft.com/en-us/library/windows/desktop/bb540537.aspx

Checklist

make -j4 test (UNIX)
tests and/or benchmarks are included
commit message follows commit guidelines

Affected core subsystem(s)

fs, doc, test

mscdex · 2017-06-22T16:55:56Z

lib/fs.js

+    if (longPath.indexOf(':', 6) !== -1) {
+      const err = new errors.Error('ERR_INVALID_WIN32_PATH', longPath);
+      if (req) {
+        return process.nextTick(function() {


I'm not sure this extra closure is needed. Perhaps it could just be process.nextTick(req.oncomplete, err).

@mscdex sometimes req.oncomplete depends on req, eg.

var context = new ReadFileContext(callback, options.encoding); context.isUserFd = isFd(path); // file descriptor ownership var req = new FSReqWrap(); req.context = context; req.oncomplete = readFileAfterOpen; ... function readFileAfterOpen(err, fd) { var context = this.context; ... }

I see. Perhaps we could at least factor out the callback then, like:

process.nextTick(onWinOpenErrorNT, req, err); // ... function onWinOpenErrorNT(req, err) { req.oncomplete(err); }

mscdex · 2017-06-22T16:57:35Z

I wonder if having two separate implementations, one for posix and one for Windows, would help any performance-wise? I think it definitely would on node v6.x, which still uses Crankshaft and would possibly allow the posix version to be inlineable.

XadillaX · 2017-06-22T17:03:22Z

@mscdex Now only one call stack is added, and an if statement added. I think it won't slow down too much.

mscdex · 2017-06-22T17:08:44Z

lib/fs.js

+  if (isWindows) {
+    // \\?\c:\somepath\1:path
+    // \\?\UNC\somepath\2:path
+    if (longPath.indexOf(':', 6) !== -1) {


If we're going to start explicitly checking for these, perhaps we should be checking for all of the reserved characters, either here in JS or at the libuv layer (via a flag of some kind if not enforced all the time for Windows).

EDIT: I just saw your comment about the other reserved characters. I'm wondering if we actually test for those on Windows nodes in CI...

silverwind · 2017-06-22T17:09:18Z

Did you test other invalid characters and/or filenames (like com, prn)? Windows has quite peculiar rules on which filenames are considered valid, ideally I think we should test them all:

https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#Naming_Conventions

XadillaX · 2017-06-22T17:11:53Z

Did you test other invalid characters and/or filenames (like com, prn)? Windows has quite peculiar rules on which filenames are considered valid, ideally I think we should test them all:

https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#Naming_Conventions

Not yet.

I will try it tomorrow, now it's 1:11 a.m. at my location and I'm going to sleep.

XadillaX · 2017-06-23T09:17:26Z

@mscdex @silverwind Sorry, I'm going to take a vacation for one week and now I don't have a Windows computer at my hand. So maybe I will continue to work for this PR after one week.

cjihrig · 2017-06-23T13:20:33Z

lib/fs.js

-               stringToFlags(options.flag || 'r'),
-               0o666,
-               req);
+  openWrap(path, stringToFlags(options.flags || 'r'), 0o666, req);


Maybe the stringToFlags() calls should be pushed into openWrap() too.

cjihrig · 2017-06-23T13:21:59Z

lib/fs.js

+    // \\?\UNC\somepath\2:path
+    if (longPath.indexOf(':', 6) !== -1) {
+      const err = new errors.Error('ERR_INVALID_WIN32_PATH', longPath);
+      if (req) {


Style nit: single line if statements shouldn't have curly braces.

cjihrig · 2017-06-23T13:23:17Z

lib/internal/errors.js

@@ -187,6 +187,7 @@ E('ERR_SOCKET_BAD_PORT', 'Port should be > 0 and < 65536');
 E('ERR_SOCKET_DGRAM_NOT_RUNNING', 'Not running');
 E('ERR_V8BREAKITERATOR', 'full ICU data not installed. ' +
  'See https://github.com/nodejs/node/wiki/Intl');
+E('ERR_INVALID_WIN32_PATH', (path) => `Invalid path under win32: '${path}'`);


I think the errors in this file are sorted by error code.

cjihrig · 2017-06-23T13:23:51Z

test/parallel/test-fs-write-file-invalid-path.js

@@ -0,0 +1,76 @@
+// Copyright Joyent, Inc. and other Node contributors.


I don't think you have to add this for new files.

cjihrig · 2017-06-23T13:24:39Z

test/parallel/test-fs-write-file-invalid-path.js

+
+const isWindows = process.platform === 'win32';
+
+if (!isWindows) return;


Can you use common.skip() here.

cjihrig · 2017-06-23T13:26:41Z

test/parallel/test-fs-write-file-invalid-path.js

+});
+
+OK.forEach((path) => {
+  fs.writeFile(path, 'world', { encoding: 'utf8' }, function(err) {


Can you add common.mustCall() to the readFile() and writeFile() callbacks in this file.

cjihrig · 2017-06-23T13:28:09Z

test/parallel/test-fs-write-file-invalid-path.js

+
+FAIL.forEach((path) => {
+  fs.readFile(path, { encoding: 'utf8' }, function(err) {
+    assert.ok(/Invalid path under win32/.test(err.message));


Please use ^ and $ to match the full error message.

silverwind · 2017-06-23T13:33:11Z

I agree with @mscdex, if we do this checking, it's better done in libuv.

Of course, it's also possible to delegate the task of checking if a filename is valid for a platform to the user, for example through modules like https://github.com/sindresorhus/valid-filename.

XadillaX · 2017-06-24T11:20:27Z

@silverwind To be honest, I think libuv's PR progress is much slower than Node.js'.

I will continue to work for other reserved names by implement a checking function next week after my vacation.

refack · 2017-06-24T21:00:55Z

I will continue to work for other reserved names by implement a checking function next week after my vacation.

Enjoy 🍹

bzoz · 2017-06-27T08:59:30Z

This file:smth is an NTFS feature, using > etc in filenames is already correctly handled by Node. Please see my comment in the original issue..

bzoz · 2017-06-27T10:50:59Z

Oh, forgot to add that I'm -1 on this.

refack · 2017-06-27T13:51:39Z

Oh, forgot to add that I'm -1 on this.

@bzoz It has an element of surprise (alternative streams I personally keep forgetting about). Should we at least document this? Or maybe add special syntax for?

bzoz · 2017-06-27T13:56:35Z

IMHO we should document this. Special syntax would be surprising for everyone that already knows this feature. It would also add maintenance burden.

XadillaX · 2017-06-27T14:03:08Z

@bzoz so what I should do is just to document it?

bzoz · 2017-06-27T14:14:13Z

I guess so.

The streams feature of NTFS is surprising, we can mention it in the docs and add a link to MSDN: https://msdn.microsoft.com/en-us/library/windows/desktop/bb540537.aspx

XadillaX · 2017-06-27T14:19:27Z

I'll update this issue to document this feature after I'm back.

XadillaX · 2017-07-03T09:40:20Z

@cjihrig, @refack, @bzoz, I've updated this PR to explain the behavior.

nodejs-github-bot added errors Issues and PRs related to JavaScript errors originated in Node.js core. fs Issues and PRs related to the fs subsystem / file system. labels Jun 22, 2017

mscdex reviewed Jun 22, 2017

View reviewed changes

cjihrig reviewed Jun 23, 2017

View reviewed changes

XadillaX force-pushed the fix/fs-colon-in-path-under-windows branch from 9498ec8 to e90f5cb Compare July 3, 2017 09:10

XadillaX changed the title ~~fs: fs.write wrong behavior under win32~~ fs,doc,test: open reserved characters under win32 Jul 3, 2017

XadillaX force-pushed the fix/fs-colon-in-path-under-windows branch from 86dc587 to c1c5d57 Compare July 3, 2017 09:43

snyk-bot mentioned this pull request Feb 12, 2022

[Snyk] Fix for 9 vulnerabilities Iceymann18777/node#10

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fs,doc,test: open reserved characters under win32 #13875

fs,doc,test: open reserved characters under win32 #13875

XadillaX commented Jun 22, 2017 •

edited

Loading

mscdex Jun 22, 2017

XadillaX Jun 22, 2017

mscdex Jun 22, 2017

mscdex commented Jun 22, 2017 •

edited

Loading

XadillaX commented Jun 22, 2017 •

edited

Loading

mscdex Jun 22, 2017 •

edited

Loading

silverwind commented Jun 22, 2017

XadillaX commented Jun 22, 2017

XadillaX commented Jun 23, 2017

cjihrig Jun 23, 2017

cjihrig Jun 23, 2017

cjihrig Jun 23, 2017

cjihrig Jun 23, 2017

cjihrig Jun 23, 2017

cjihrig Jun 23, 2017

cjihrig Jun 23, 2017

silverwind commented Jun 23, 2017

XadillaX commented Jun 24, 2017

refack commented Jun 24, 2017

bzoz commented Jun 27, 2017

bzoz commented Jun 27, 2017

refack commented Jun 27, 2017

bzoz commented Jun 27, 2017

XadillaX commented Jun 27, 2017

bzoz commented Jun 27, 2017

XadillaX commented Jun 27, 2017

XadillaX commented Jul 3, 2017

		@@ -0,0 +1,76 @@
		// Copyright Joyent, Inc. and other Node contributors.


		const isWindows = process.platform === 'win32';

		if (!isWindows) return;

fs,doc,test: open reserved characters under win32 #13875

fs,doc,test: open reserved characters under win32 #13875

Conversation

XadillaX commented Jun 22, 2017 • edited Loading

Checklist

Affected core subsystem(s)

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mscdex commented Jun 22, 2017 • edited Loading

XadillaX commented Jun 22, 2017 • edited Loading

mscdex Jun 22, 2017 • edited Loading

Choose a reason for hiding this comment

silverwind commented Jun 22, 2017

XadillaX commented Jun 22, 2017

XadillaX commented Jun 23, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

silverwind commented Jun 23, 2017

XadillaX commented Jun 24, 2017

refack commented Jun 24, 2017

bzoz commented Jun 27, 2017

bzoz commented Jun 27, 2017

refack commented Jun 27, 2017

bzoz commented Jun 27, 2017

XadillaX commented Jun 27, 2017

bzoz commented Jun 27, 2017

XadillaX commented Jun 27, 2017

XadillaX commented Jul 3, 2017

XadillaX commented Jun 22, 2017 •

edited

Loading

mscdex commented Jun 22, 2017 •

edited

Loading

XadillaX commented Jun 22, 2017 •

edited

Loading

mscdex Jun 22, 2017 •

edited

Loading