Should Stream.pipe() call resume() automatically? (Maybe just for stdin?)

[isaacs]

It would be nice if this:

process.stdin.pipe(process.stdout);

were equivalent to cat. However, since stdin starts out in a paused state, this does nothing, and you have to do:

process.stdin.pipe(process.stdout);
process.stdin.resume();

which is 100% more lines of code. I find that rather objectionable.

Options:

1. Stream.prototype.pipe should call this.resume() after setting up the destination. This would be the fix for other streams exhibiting the same problem, but would have much further reaching side effects. I'm very wary about doing this for 0.8, but it could be explored for 0.9.
2. process.stdin.pipe should call this.resume() after applying Stream.prototype.pipe. This would be the surgical fix, but is perhaps still too hazardous to do now for 0.8 without exploring edge cases.

[isaacs]

Opinions from heavy Stream users please: @ry @mikeal @felixge @substack @dominictarr @maxogden

[dominictarr]

I think this is reasonable, but if it did, it should do so at nextTick at the earliest.
It's possible (likely, in my code) that I'm gonna pipe several.pipe(streams).pipe(together).

It's likely that the first stream is sync (it might just parse or resplit the chunks into lines or something).
so I don't want stdin to start spewing out events until after the last pipe call has returned.

calling resume on the next line is a good bet for this, but hard to remember to do for some streams but not others.

I've written and used streams that start the tick after pipe and found it pretty intuitive.

[max-mapper]

as long as you can still pause a new steam and do buffering magics then this seems like a nice default behavior

[evlun]

I'm with @dominictarr on this one.

How about extending the options argument? So, something like:

// this will invoke b.resume() on the next tick
a.pipe(b);

// ...while this won't
a.pipe(b, { 'resume': false });

[mikeal]

-1 on adding options.

.pipe() should call resume but resume() MUST do nothing until nextTick(). This way if it is to be paused again before nextTick() it'll go back in to it's paused state.

i think this will also be effected by something isaac and i talked about a while back that "streams should open in a paused state and be resumed before the emit/accept data". this solves the problems we have now with streams essentially opening a closed state until they can emit/accept data.

-Mikeal

On Jun 21, 2012, at June 21, 201211:12 AM, Erik Lundin wrote:

I'm with @dominictarr on this one.

How about extending the options argument? So, something like:

// this will invoke b.resume() on the next tick
a.pipe(b);

// ...while this won't
a.pipe(b, { 'resume': false });

---

Reply to this email directly or view it on GitHub:
#3503 (comment)

[mikeal]

also, I find this less intuitive:

stream.pipe(writable, {resume:false})

than this

stream.pipe(writable)
stream.pause()

The first requires more knowledge about stream internals, the second one matches the intuitive behavior of the public API.

[evlun]

I agree that the dest.resume() call should be canceled if dest.pause() is invoked before the next tick.

I personally don't think it's too much to assume that the user has some knowledge of how .pipe(dest) works. However, if intuitiveness is a major concern, I don't think the use of dest.pause() will be very straight forward to new users. Using the example provided by @mikeal:

stream.pipe(writable)
stream.pause()

Someone who doesn't know that .pipe(dest) waits a tick before invoking .resume() on the destination object might think that any "buffered" data will be flushed to writable before the stream is paused again.

[isaacs]

@mikeal Your "more intuitive" approach would work as expected if stream.pipe(writable) did resume by default. (At least, on process.stdin, which is the only stream in Node that starts out paused.)

[mikeal]

@MJor and all.

Note: I think you have a typo, it's the src not the dest that is getting resumed.

I'm making 2 assumptions here:

1. that .resume() on pipe() is the default behavior. we are presumably doing this because we think it's what users most likely want to do the majority of the time.
2. that we don't emit events until nextTick() and calling .pause() before nextTick() will cancel the resume and leave the stream in a paused state.

Now, if a user pipes a stream to a destination and notices it begins emitting events I doubt that, without intimate knowledge of the stream implementation, they would assume resume() is getting called as a result of pipe(). They could assume all kinds of behavior: perhaps the stream didn't really open until it was piped or perhaps someone else resumed it.

The user might make any number of assumptions about the underlying implementation and assuming they would search or find or guess that setting {resume:false} would fix their issue seems far fetched. What they are most likely to do is simply pause() the stream.

@isaacs

right, so what happens when that stream opens paused and gets resumed by a pipe call before it is actually open? i doubt it's a spec issue, it's something to think about in implementation tho.