cmd: support dash as stdin alias #13012

ebraminio · 2017-05-13T08:14:58Z

make -j4 test (UNIX), or vcbuild test (Windows) passes
(some of the tests are failing regardless of my change AFAICS so I assume this as passed)
tests and/or benchmarks are included
(an available test is modified to cover this, and a more specific one also added)
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)

cli

Dash as stdin alias is a usual convention between unix programs https://unix.stackexchange.com/a/16364 and I believe it should be available on nodejs also.

Here I've made a change that make node ignore dash as an option so it will effectively be treated like /dev/stdin

Simply, with my change this will be made possible with node:

echo "console.log(process.argv[2])" | node - --option-for-stdin-script

The reason I like to have this is because of my one-file webserver http://pad.js.org which in order to be able to pass extra option to its one-linear execution command on the fly (without having to download or install the script first, which it supports that also through npm and docker). BTW, have a look at that project also, I guess you will find it interesting, specially the way its source is distributed and several features it offers :)

Interestingly, I saw during my tests if I pass "/std/stdin" instead "-" on the above command, even without my change, it will be ran on macOS just fine but for some reasons it seems it doesn't work on Linux equally which perhaps should be filed as a bug separately (Edited: proposed another patch for that). However with this change, "-" will provide a cross platform solution for achieving this goal.

For my specific case, this patch will make this functionality possible:

curl pad.js.org | node - -h

which as mentioned before if you have access to macOS, this is already possible with the following command, but not on Linux and obviously ever Windows:

curl pad.js.org | node /dev/stdin -h

Thanks.

refack · 2017-05-13T13:25:39Z

test/parallel/test-stdin-script-child.js

+  setTimeout(function() {
+    child.stdin.end('console.log(process.pid)');
+  }, 1);
+}


Nit: We like an \n on the last line

Done for both

refack · 2017-05-13T17:56:08Z

test/parallel/test-stdin-script-child.js

-let found = '';
+for (const args of [[], ['-']]) {
+  const child = spawn(process.execPath, args, {
+    env: Object.assign(process.env, {


Don't you want the arguments to assign the other way around?

This is just copied… is this an existing bug? :/

This is just copied from the old version, I guess better to not touch it now.

🤔 this is a super weird construct. IMHO If it's to suss out a bug it should be near https://github.com/nodejs/node/blob/master/test/parallel/test-child-process-spawn-shell.js#L51
Personaly I'd rather see this fixed here, but I won't block for it.

refack · 2017-05-13T18:00:19Z

lib/internal/bootstrap_node.js

@@ -123,7 +123,7 @@
        const internalModule = NativeModule.require('internal/module');
        internalModule.addBuiltinLibsToObject(global);
        evalScript('[eval]');
-      } else if (process.argv[1]) {
+      } else if (process.argv[1] && process.argv[1] !== '-') {


IMHO -- is much more common alias for "the rest will be piped through stdin". If it's not too much work I'd be happy if you adjust.
So you could do echo "console.log(process.argv[2])" | node -h --

Got you -h is an arg to the script. Not to node.
Retracting.

We already use -- for separating arguments, and that’s what it’s commonly used for (on Unixes, at least). - is the most common choice for “stdin”, by far.

refack

Last comment is a real nit.

addaleax · 2017-05-13T18:09:14Z

src/node.cc

-  while (index < nargs && argv[index][0] == '-' && !short_circuit) {
+  while (index < nargs &&
+         // starts with dash but is not only a dash which is special
+         argv[index][0] == '-' && strlen(argv[index]) != 1 &&


you could replace the strlen check with argv[index][1] == '\0'

addaleax · 2017-05-13T18:09:50Z

test/parallel/test-stdin-script-child.js

-let found = '';
+for (const args of [[], ['-']]) {
+  const child = spawn(process.execPath, args, {
+    env: Object.assign(process.env, {


This is just copied… is this an existing bug? :/

refack · 2017-05-13T18:10:18Z

CI: https://ci.nodejs.org/job/node-test-commit/9868/

ebraminio · 2017-05-13T18:23:59Z

I think I've fixed these

refack · 2017-05-13T18:45:49Z

New CI: https://ci.nodejs.org/job/node-test-commit/9869/

ebraminio · 2017-05-13T18:50:15Z

Thank you :) Is linter giving a true found error? I couldn't find anything useful from its console info.

refack · 2017-05-13T19:02:37Z

test/parallel/test-stdin-script-child.js

+  setTimeout(function() {
+    child.stdin.end('console.log(process.pid)');
+  }, 1);
+}


Same here just one blank line, i.e. file should end }\n

Super weird, I really didn't want to put two... I guess done now

ebraminio · 2017-05-13T19:19:04Z

Great, it seems CI don't dislike this anymore

refack · 2017-05-13T19:20:28Z

Yeah I run a linter CI: https://ci.nodejs.org/job/node-test-linter/9043/

ebraminio · 2017-05-13T19:48:05Z

No meaningful output on test/linux-fips AFAICS

refack · 2017-05-13T21:17:36Z

I've retriggered linux-fips: https://ci.nodejs.org/job/node-test-commit-linux-fips/8382/
But even if it borks again, consider the CI green. Now we wait for this to "mature"...

I'm assuming this is your first contribution so I've put down a little explanation. If it's not ignore the rest 😉

Our "landing" policy is to wait 2 days (3-4 if it's weekend) so that stakeholders can chime in. Since I've marked this as semver-minor there shouldn't be any extra steps. If someone else judges this as semver-major then we need two members of the CTC to sign off on the change.

Good luck, and thank you very much for the contribution 🥇

ebraminio · 2017-05-14T06:17:30Z

np, I just wanted to know if my part is complete :)

addaleax · 2017-05-14T11:48:41Z

src/node.cc

@@ -3780,7 +3780,10 @@ static void ParseArgs(int* argc,

  unsigned int index = 1;
  bool short_circuit = false;
-  while (index < nargs && argv[index][0] == '-' && !short_circuit) {
+  while (index < nargs &&
+         // starts with dash but is not only a dash which is special


I’d drop this comment, or change it to something like // skip "-", because currently it described how it does stuff, not what it does, so it’s directly redundant to the code.

addaleax · 2017-05-14T11:49:43Z

test/parallel/test-stdin-script-child-option.js

+child.stdin.end('console.log(process.argv[2])');
+
+let actual = '';
+child.stdout.on('data', (chunk) => actual += chunk);


You should call child.stdout.setEncoding('utf8'); before using .on('data') with strings

ebraminio · 2017-05-14T17:14:57Z

Done

Fishrock123 · 2017-05-19T14:40:43Z

Going to land this at the end of today, then.

ebraminio · 2017-05-20T13:22:10Z

src/node.cc

@@ -3598,6 +3598,8 @@ static void PrintHelp() {
         "                             does not appear to be a terminal\n"
         "  -r, --require              module to preload (option can be "
         "repeated)\n"
+         "  -                          script read from stdin (default; "
+         "interactive mode if a tty)"


Just added this line from the python -h, please review it before the merge.

~~You mean node -h?~~ ahh "from"

I meant I've copied this literally from the similar place on python -h :)

ebraminio · 2017-05-20T13:24:37Z

doc/api/cli.md

@@ -10,7 +10,7 @@ To view this documentation as a manual page in a terminal, run `man node`.

 ## Synopsis

-`node [options] [v8 options] [script.js | -e "script"] [--] [arguments]`
+`node [options] [v8 options] [script.js | -e "script" | -] [--] [arguments]`


Removed space after the dash as its section start also is not space padded.

jasnell · 2017-05-22T15:20:33Z

test/parallel/test-stdin-script-child.js

+  child.on('close', common.mustCall(function(c) {
+    assert.strictEqual(c, 0);
+    assert.strictEqual(found, wanted);
+    console.log('ok');


Is the console output necessary to the test? If not, please remove :-)

refack · 2017-05-23T02:46:38Z

Going to land this at the end of today, then.

@Fishrock123 are you still planning on landing this?

addaleax · 2017-05-23T18:44:31Z

Landed in 594b5d7, thanks for the PR!

PR-URL: #13012 Reviewed-By: Jeremiah Senkpiel <[email protected]> Reviewed-By: Refael Ackermann <[email protected]> Reviewed-By: Gibson Fahnestock <[email protected]>

Fishrock123 · 2017-05-23T23:36:22Z

@refack ok not really, go ahead and land it please

Fishrock123 · 2017-05-23T23:36:43Z

huh, guess github didn't update...

Thanks @addaleax 😬 😓

PR-URL: #13012 Reviewed-By: Jeremiah Senkpiel <[email protected]> Reviewed-By: Refael Ackermann <[email protected]> Reviewed-By: Gibson Fahnestock <[email protected]>

gibfahn · 2018-01-15T21:34:34Z

Release team were +1 on backporting to v6.x if someone would be willing to raise a backport PR, see the guide for more info.

silverwind · 2018-01-29T20:34:39Z

If this gets backported, #18440 should probably also be applied.

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. labels May 13, 2017

mscdex added cli Issues and PRs related to the Node.js command line interface. and removed c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. labels May 13, 2017

ebraminio force-pushed the master branch from 3c16607 to d96acd0 Compare May 13, 2017 16:39

refack approved these changes May 13, 2017

View reviewed changes

refack suggested changes May 13, 2017

View reviewed changes

addaleax reviewed May 13, 2017

View reviewed changes

ebraminio force-pushed the master branch from d96acd0 to eae0048 Compare May 13, 2017 18:19

refack approved these changes May 13, 2017

View reviewed changes

refack suggested changes May 13, 2017

View reviewed changes

ebraminio force-pushed the master branch 2 times, most recently from eefb4e3 to 1497e2b Compare May 13, 2017 19:08

refack approved these changes May 13, 2017

View reviewed changes

refack added the semver-minor PRs that contain new features and should be released in the next minor version. label May 13, 2017

refack self-assigned this May 13, 2017

addaleax reviewed May 14, 2017

View reviewed changes

ebraminio force-pushed the master branch from 1497e2b to 2e9ed1d Compare May 14, 2017 17:10

ebraminio mentioned this pull request May 14, 2017

fs: fix realpath{Sync} on resolving pipes/sockets #13028

Closed

4 tasks

gibfahn approved these changes May 18, 2017

View reviewed changes

Fishrock123 added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. labels May 19, 2017

ebraminio force-pushed the master branch 3 times, most recently from 877fbfe to 0283fc0 Compare May 20, 2017 13:21

ebraminio commented May 20, 2017

View reviewed changes

ebraminio mentioned this pull request May 20, 2017

cli: accept /dev/stdin as input file on Linux #13130

Closed

3 tasks

refack removed their assignment May 20, 2017

jasnell reviewed May 22, 2017

View reviewed changes

ebraminio force-pushed the master branch 2 times, most recently from 3a63d34 to 55e51d6 Compare May 22, 2017 17:44

cmd: support dash as stdin alias

ebfaead

ebraminio force-pushed the master branch from 55e51d6 to ebfaead Compare May 22, 2017 17:51

addaleax closed this May 23, 2017

addaleax pushed a commit that referenced this pull request May 23, 2017

cmd: support dash as stdin alias

594b5d7

PR-URL: #13012 Reviewed-By: Jeremiah Senkpiel <[email protected]> Reviewed-By: Refael Ackermann <[email protected]> Reviewed-By: Gibson Fahnestock <[email protected]>

jasnell pushed a commit that referenced this pull request May 24, 2017

cmd: support dash as stdin alias

a834733

PR-URL: #13012 Reviewed-By: Jeremiah Senkpiel <[email protected]> Reviewed-By: Refael Ackermann <[email protected]> Reviewed-By: Gibson Fahnestock <[email protected]>

jasnell pushed a commit that referenced this pull request May 28, 2017

cmd: support dash as stdin alias

f146fe4

PR-URL: #13012 Reviewed-By: Jeremiah Senkpiel <[email protected]> Reviewed-By: Refael Ackermann <[email protected]> Reviewed-By: Gibson Fahnestock <[email protected]>

jasnell mentioned this pull request May 28, 2017

8.0.0 Release Proposal #12220

Closed

vsemozhetbyt mentioned this pull request May 30, 2017

blog: add release post for 8.0.0 nodejs/nodejs.org#1241

Closed

gibfahn added the lts-watch-v6.x label Jan 15, 2018

ebraminio mentioned this pull request Jun 24, 2018

Treat - just as /dev/stdin and remove one extra file reader harfbuzz/harfbuzz#1065

Merged

cmd: support dash as stdin alias #13012

cmd: support dash as stdin alias #13012

Conversation

ebraminio commented May 13, 2017 • edited Loading

Affected core subsystem(s)

refack May 13, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

refack May 13, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

refack May 13, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

refack left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

refack commented May 13, 2017

ebraminio commented May 13, 2017

refack commented May 13, 2017

ebraminio commented May 13, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ebraminio commented May 13, 2017

refack commented May 13, 2017

ebraminio commented May 13, 2017

refack commented May 13, 2017

ebraminio commented May 14, 2017

addaleax May 14, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ebraminio commented May 14, 2017

Fishrock123 commented May 19, 2017

Choose a reason for hiding this comment

refack May 20, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ebraminio May 20, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

refack commented May 23, 2017

addaleax commented May 23, 2017

Fishrock123 commented May 23, 2017

Fishrock123 commented May 23, 2017 • edited Loading

gibfahn commented Jan 15, 2018

silverwind commented Jan 29, 2018

ebraminio commented May 13, 2017 •

edited

Loading

refack May 13, 2017 •

edited

Loading

refack May 13, 2017 •

edited

Loading

refack May 13, 2017 •

edited

Loading

addaleax May 14, 2017 •

edited

Loading

refack May 20, 2017 •

edited

Loading

ebraminio May 20, 2017 •

edited

Loading

Fishrock123 commented May 23, 2017 •

edited

Loading