v6.13.1 createReadStream() invalid seek

[wankdanker]

• Version: v6.13.1
• Platform: Linux 3.13.0-142-generic doc: util: document --trace-deprecation #191-Ubuntu SMP Fri Feb 2 12:14:37 UTC 2018 i686 i686 i686 GNU/Linux
• Subsystem: fs

It believe commit 82bdf8f (fs: fix options.end of fs.ReadStream()) broke my code for creating a read stream. It looks to me like this.start is now forced to the value 0 whereas previously it was allowed to be undefined. I'm guessing since it is 0 it now forces a seek operation which breaks my use case.

In v6.13.1, this code:

var fs = require('fs');
var reader = fs.createReadStream('/dev/ttyS0');

results in this error:

Error: ESPIPE: invalid seek, read
    at Error (native)

The above code in v6.13.0 does not create that error.

In v6.13.0, this code:

var fs = require('fs');
var reader = fs.createReadStream('/dev/ttyS0', { start : 0 });

results in this error:

Error: ESPIPE: invalid seek, read
    at Error (native)

[MylesBorins]

/cc @MoonBall @jasnell @cjihrig @joyeecheung

[MoonBall]

@wankdanker I'm sorry for it. But I think current code is ok since we considered that if fd is specified and start is omitted or undefined, fs.createReadStream() reads sequentially from the current file position.

I think your problem will be solved by getting the fd of '/dev/ttyS0'. You can try the below code, but I'm not sure it's going to succeed.

const fs = require('fs');
const util = require('util');

util.promisify(fs.open)('/dev/ttyS0', 'r').then(fd => {
  const reader = fs.createReadStream(undefined, { fd: fd });
  reader.on('data', console.log);
});

[wankdanker]

@MoonBall Thanks for your reply. I've had this code in production since at least as far back as v0.10.x without this breaking. Just seems odd to me to see this changed all of a sudden in v6.13.1.

[MoonBall]

@wankdanker I suppose, the cause is the /dev/ttyS0 file doesn't support seeking. But I didn't find an official document to explain it.

[MylesBorins]

@MoonBall a semverpatch should never change behavior, we may want to revert this on all LTS lines and mark it as semver-major for 10.x. Even if the behavior is incorrect we might not want to change it in LTS releases if people are relying on it.

/cc @nodejs/lts

[MoonBall]

@MylesBorins reverting it is ok to me. #18121 just fixed a small issue and I didn't consider this problem.

[MylesBorins]

@wankdanker I've opened a reversion against v6.x and am in the process of building a test release. Once that release is built would you be able to run it and verify in the PR that this fixes your problem

[MylesBorins]

@wankdanker I've tried to reproduce on OSX

running

var fs = require('fs');
var reader = fs.createReadStream('/dev/ttyS0');

results in the error you report for v6.13.1 on various version of v6.x including v6.13.0 and v6.12.3

I'm going to test on a cloud ubuntu box and see if I can repro.

Can you tell us a bit more about your setup

edit: looks like there is no way for me to test this in the cloud rn

[addaleax]

@MylesBorins Hope it doesn’t bother you that I tend to chime in on these kinds of issues :) This problem is not specific to v6.x, so I think we should take care of it in master too.

Reverting would be an option, but I think we could also fix the underlying issue here, maybe in a better way than before:

diff in the fold

diff --git a/lib/fs.js b/lib/fs.js
index f890e431d2a9..8324ff0542a9 100644
--- a/lib/fs.js
+++ b/lib/fs.js
@@ -1967,8 +1967,7 @@ function ReadStream(path, options) {
   this.flags = options.flags === undefined ? 'r' : options.flags;
   this.mode = options.mode === undefined ? 0o666 : options.mode;
 
-  this.start = typeof this.fd !== 'number' && options.start === undefined ?
-    0 : options.start;
+  this.start = options.start;
   this.end = options.end;
   this.autoClose = options.autoClose === undefined ? true : options.autoClose;
   this.pos = undefined;
@@ -1993,6 +1992,12 @@ function ReadStream(path, options) {
     this.pos = this.start;
   }
 
+  // Backwards compatibility: Make sure `end` is a number regardless of `start`.
+  // TODO(addaleax): Make the above typecheck not depend on `start` instead.
+  // (That is a semver-major change).
+  if (typeof this.end !== 'number')
+    this.end = Infinity;
+
   if (typeof this.fd !== 'number')
     this.open();
 
@@ -2047,6 +2052,8 @@ ReadStream.prototype._read = function(n) {
 
   if (this.pos !== undefined)
     toRead = Math.min(this.end - this.pos + 1, toRead);
+  else
+    toRead = Math.min(this.end - this.bytesRead + 1, toRead);
 
   // already read everything we were supposed to read!
   // treat as EOF.
diff --git a/test/parallel/test-fs-read-stream.js b/test/parallel/test-fs-read-stream.js
index 7fc7a0d56bce..75b2fe3d14b8 100644
--- a/test/parallel/test-fs-read-stream.js
+++ b/test/parallel/test-fs-read-stream.js
@@ -21,7 +21,9 @@
 
 'use strict';
 const common = require('../common');
+const tmpdir = require('../common/tmpdir');
 
+const child_process = require('child_process');
 const assert = require('assert');
 const fs = require('fs');
 const fixtures = require('../common/fixtures');
@@ -178,6 +180,31 @@ common.expectsError(
   }));
 }
 
+{
+  // Verify that end works when start is not specified, and we do not try to
+  // use positioned reads. This makes sure that this keeps working for
+  // non-seekable file descriptors.
+  tmpdir.refresh();
+  const filename = `${tmpdir.path}/foo.pipe`;
+  const mkfifoResult = child_process.spawnSync('mkfifo', [filename]);
+  if (!mkfifoResult.error) {
+    child_process.exec(`echo "xyz foobar" > '${filename}'`);
+    const stream = new fs.createReadStream(filename, { end: 1 });
+    stream.data = '';
+
+    stream.on('data', function(chunk) {
+      stream.data += chunk;
+    });
+
+    stream.on('end', common.mustCall(function() {
+      assert.strictEqual('xy', stream.data);
+      fs.unlinkSync(filename);
+    }));
+  } else {
+    common.printSkipMessage('mkfifo not available');
+  }
+}
+
 {
   // pause and then resume immediately.
   const pauseRes = fs.createReadStream(rangeFile);

Will open a PR shortly if that’s okay.

[MylesBorins]

Here is a link to the 8.x revert #19328

@addaleax I am absolutely thrilled when you chime in. I push for a revert only because I lack time + context to solve this in a more elegant way. I 100% support moving forward with a better solution if we can fix it properly!

[wankdanker]

Hi @MylesBorins!

Sure! We've got several machines running Ubuntu 14.04 and 16.04 that have Mettler Toledo scales connected to the serial port. It's been a long time since I wrote my application and worked with the scale protocol, but I believe until we write to the serial port asking the scale for a measurement, there is nothing to read from the serial port.

Let me know if you have any other questions. And I'll run the test build whenever that is available. Thanks!

[wankdanker]

@MylesBorins The test build worked for me. Thank you.

[addaleax]

@wankdanker I don’t know if trying it out is an option for you, but I’m pretty confident that #19329 fixes this issue without undoing the original fix, so there’s a decent chance we’ll be going with that.

If you want to & can try that, that would be awesome. I suppose that if it’s much more convenient for you, Myles or somebody else from the release team could also produce another test build with it.

[wankdanker]

@addaleax If I get a chance, I'll give it a go. It would definitely easier if someone generated a test build, but I wouldn't want to inconvenience anyone. 😄

[gibfahn]

Hope it doesn’t bother you that I tend to chime in on these kinds of issues :)

@addaleax I am absolutely thrilled when you chime in. I push for a revert only because I lack time + context to solve this in a more elegant way. I 100% support moving forward with a better solution if we can fix it properly!

💯 to what Myles said (from an 8.x perspective). Extremely grateful for anyone who is willing to get involved and work out a better solution than just reverting.

[fivdi]

I'm seeing the same issue with Node v8.10.0 when using read streams to read from FIFO special files (named pipes) on Linux. With Node v8.9.4 all is ok.

[addaleax]

Opened backports for the fixes in #19410 and #19411.

@nodejs/lts Given the comments here, do we want to consider fast-tracking those?

[gibfahn]

@nodejs/lts Given the comments here, do we want to consider fast-tracking those?

I think given that we were willing to revert in the next release, then we should fast-track this (given that we're not reverting).