Skip to content

added initial draft for stream interfaces #55

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 28, 2016
Merged

added initial draft for stream interfaces #55

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
45fa997
added initial draft for stream interfaces
bergos Mar 18, 2016
e871c56
replaced read/write with match/import
bergos Apr 26, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions interface-spec.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,3 +117,56 @@ Triples always have `.graph` set to DefaultGraph.
- `DefaultGraph .defaultGraph()` returns an instance of DefaultGraph.
- `Triple .triple(Term subject, Term predicate, Term object, [Term graph])` returns a new instance of Triple.
- `Quad .quad(Term subject, Term predicate, Term object, [Term graph])` returns a new instance of Quad.

## Stream interfaces

Streams are used only in a readable manner.
This requires only a single queue per stream, which simplifies implementations and doesn't have performance drawbacks, compared to writeable streams.

### Stream extends EventEmitter

**Methods:**

- `Quad .read()`
This method pulls a quad out of the internal buffer and returns it.
If there is no quad available, then it will return null.

**Events:**

- `readable`
Copy link
Contributor

@pieroit pieroit Apr 1, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about start?

Copy link
Contributor

@dmitrizagidulin dmitrizagidulin Apr 1, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to start instead of readable

Copy link
Member Author

@bergos bergos Apr 22, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

readable is an official Node.js event. So let's stay with that standard.

When a quad can be read from the stream, it will emit this event.

- `end`
This event fires when there will be no more quads to read.

- `error`
This event fires if any error occurs.
The error message is forwarded to the event listener.

- `data`
This event is emitted for every quad that can be read from the stream.
The quad is forwarded to the event listener.

### Source

- `Stream .read([Term|RegExp subject], [Term|RegExp predicate], [Term|RegExp object], [Term|RegExp graph])`
Copy link
Member Author

@bergos bergos Mar 21, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See .write method of Sink

Copy link
Member

@RubenVerborgh RubenVerborgh Mar 24, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should become something like search. I expect read to be an operation without parameters, like in Stream itself.

Copy link
Member Author

@bergos bergos Mar 28, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the last meeting we collected the following alternative methods names. What do you think about them? Other proposals?

.search
.filter // may suggest accepting filter function
.match // used by rdf-interfaces

Copy link
Member

@elf-pavlik elf-pavlik Mar 29, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 .match
Store has .removeMatches() with the same signature, so .match() or even .getMaches() looks consistent.

Copy link
Member Author

@bergos bergos Apr 1, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 .match
-1 .filter

Copy link
Member Author

@bergos bergos Apr 22, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have at least 2 * +1 for .match. I will change my PR. If there are good arguments for a different method name, we can change that later.

Returns a stream that processes all quads matching the pattern.

### Sink

- `undefined .write(Stream stream)`
Copy link
Member Author

@bergos bergos Mar 21, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The .write method could be confusing. It's more like a reverse .pipe, but that method name should be reserved, because some classes may implemented full featured streams. Another proposed method name was .consume.

It's the same for .read. Reading triples from a source could be also described as writing all triples from the source to a stream. .filter could be used, but would be confusing if the filter feature isn't used.

Copy link
Member Author

@bergos bergos Mar 28, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the last meeting we collected the following alternative methods names. What do you think about them? Other proposals?

.accept
.consume
.input
.import
.append
.readFrom
.pull(From)
.reversePipe

Copy link
Member

@RubenVerborgh RubenVerborgh Apr 1, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for import/readFrom

-1 for pull (without From)/reversePipe

Copy link
Member Author

@bergos bergos Apr 1, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 .consume / .import
-1 .accept / .input / .readFrom / .pull / .pullFrom / .reversePipe

Copy link
Contributor

@pieroit pieroit Apr 1, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 .consume / .accept / .readFrom / .reversePipe / .append

Copy link

@dlongley dlongley Apr 1, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I strongly recommend cribbing from:

https://nodejs.org/api/stream.html#stream_event_readable
https://streams.spec.whatwg.org/

Copy link
Member

@RubenVerborgh RubenVerborgh Apr 2, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dlongley We're indeed using a subset of Node.js streams (intentionally not the full interface, because its backpressure control makes it expensive).

Copy link
Member Author

@bergos bergos Apr 22, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have at least 2 * +1 for .import. I will change my PR. If there are good arguments for a different method name, we can change that later.

Writes all quads from the stream to the sink.

### Store extends Source, Sink

- `EventEmitter .remove(Stream stream)`
Removes all streamed quads.
The `end` and `error` events are used like described in the `Stream` interface.

- `EventEmitter .removeMatches([Term|RegExp subject], [Term|RegExp predicate], [Term|RegExp object], [Term|RegExp graph])`
All quads matching the pattern will be removed.
The `end` and `error` events are used like described in the `Stream` interface.

- `EventEmitter .deleteGraph(IRI|String graph)`
Deletes the given named graph.
The `end` and `error` events are used like described in the `Stream` interface.