feat: parameterise resources

[pemrouz]

Consider using @sammyt's rhumb library for more RESTful style resource identification. Currently we are bombing all resources to all nodes. This was restricted per-resource with marking some resources as private. However, the mobile clients (@gangji91, @3liv) are struggling with loading entire collections. After parameterisation, we should also look to implement backpressure. So, instead of:

<twitter-feed data="tweets.data" />

It would look more like:

<twitter-feed data="/tweets/latest/10" >

or

ripple('/tweets/latest/10')

(I was planning on dropping the requirement to have the type in the name already).

This example makes me wonder though whether we can do more with implementation using streams - which would enable more FRP style components. Or would it complicate them..

[mstade]

This example makes me wonder though whether we can do more with implementation using streams - which would enable more FRP style components. Or would it complicate them..

Wat?

[pemrouz]

Yes Wat!

(Note: will post something more relevant in the morning :), just about to knock out)..

Sent from my iPhone

On 4 Dec 2014, at 01:56, Marcus Stade [email protected] wrote:

This example makes me wonder though whether we can do more with implementation using streams - which would enable more FRP style components. Or would it complicate them..

Wat?

—
Reply to this email directly or view it on GitHub.

[pemrouz]

@mstade: To elaborate on that, the core assumption is that components all work as f(data), where 'data' has so far been a snapshot in time of the data. When the data changes, it is re-invoked with the latest. But I'm thinking what's passed in could be a live stream as opposed to a snapshot in time. I just wasn't sure if that would break the whole simple paradigm, since we re-invoke the function with the new data on change events, rather than the data itself receiving change events. I was in two minds about whether it could work..

[mstade]

What's a stream? What does it look like?

[pemrouz]

http://nodejs.org/api/stream.html

[mstade]

That's my point, actually. Streams in all their incarnations in js land are just event emitters, and given components are effectively event handlers, why add more to the mix? You already have a stream, is what I'm saying.

[pemrouz]

That's my point, actually. Streams in all their incarnations in js land are just event emitters, and given components are effectively event handlers, why add more to the mix? You already have a stream, is what I'm saying.

👍 :)

[pemrouz]

Ok, so some more thoughts: Thus far I've been freestyling a bit (albeit, consistently) with terminology around resources. It's important to tidy this up as it will impact the structure/extension of the project. Looking around, there is a few terms I'd like to introduce: URI, URN, URC, URL. I'm aware some of these have fallen out of fashion, but the distinctions are conceptually useful for discussion. We have today:

// Definition
ripple.resource('{urn}', {resource}, {urc})

// Declarative Invocation
<{urn1} data='{urn2}' />

// Imperative Invocation
ripple('{urn}')                               // i. synchronous
ripple('{urn}').on('response', function(){ }) // ii. asynchronous

where:

• URN: is a "globally unique, persistent identifier used for recognition, for access to characteristics of the resource or for access to the resource itself".. "providing a mechanism for the identification of resources in particular namespaces".. "even when the resource ceases to exist or becomes unavailable" [RFC1737]. URN seems to be the most accurate here (vs URL, URI), as a "URN defines an item's identity, while the URL provides a method for finding it". I do not mean to imply here that a URN does/should follow the deprecated naming convention (prefixed with urn:{nid}:{nss} etc).
• URC: is metadata for a resource. Currently, I'm using "headers" to describe this, but URC seems better to describe permanent resource metadata "at rest", whilst "headers" implies to me metadata in the context of a message transfer between a particular sender/receiver.

Now, in this light, there are two options (note: "classical vs contemporary" reference to W3C Note 21):

Classical

If we were to stick to the classical approach (uri=(urn+urc)||url), we would use the URC to "bind a URI's associated URN to its URL(s)":

// Definition
ripple.resource('{urn}', {resource}, { url: ['/route1/:param', '/route2/:param'] })

A handler (perhaps, per-URL handler) would be invoked everytime /route1 or /route2 is invoked.

Contemporary

The contemporary understanding acknowledges that a URI can be a URN, URL or both.

Currently, we can already create a "hollow resource". In the following example, "users" will be populated from a database, but the "colleagues" resource body will always be empty on the server. Using the proxies, when the body is changed (from client) we do nothing (return false), and when responding to a particular request (to client) we filter all users down to the ones in the same team as the requestor:

ripple
  .resource('users', [])
  .resource('colleagues', [], { from: from, to: to })

function from(req){
  return false
}

funtionc to(res){
  return ripple('users')
    .filter(by('team', res.currentUser.team))
}

We can extend this slightly, to allow parameterised names (routes):

ripple
  .resource('tweets', [])
  .resource('/tweets/latest/:quantity', function(req, res){ .. })

Where a route is defined (leading slash), it will internally be defined similar to a hollow resource. It will have access to parameters under req.params and will just need return a value, or use res in advanced cases, such as responding to just a subset of clients (res.emit(res.sessionID)(value)). This is essentially REST over WebSockets.

So the final technical definition would be:

ripple
  .resource('{uri}', {body}, {urc}) // actual resource
  .resource('{uri}', {handler})     // route, hollow resource

Bonus: Namespaces

We only support one-to-many server-client relationships (many databases is conceptually not hard to do with similar proxies/hooks). This means that one server de facto defines one namespace, one web boundary of resources. If we allowed sever nodes to communicate, how would we demarcate namespaces? More importantly, how can this be achieved in a decentralised fashion? This is where the allure of URN's comes back over URL's..

// somewhere in Manchester..
ripple
  .resource('urn:nhs:patients', ..)

// somewhere in London..
ripple
  .resource('urn:nhs:patients', ..)

// then, some GP-hacker in Scotland should be able to enter in devtools:
ripple('urn:nhs:patients:1') // record for patient 1
ripple('urn:nhs:patients:2') // record for patient 2

But, I haven't thought much about decentralised resource definition/resolution yet..

Any thoughts/totally different perspectives how realtime resources should be defined/other challenges? @mstade @sammyt @mamapitufo @gabrielmontagne @tomsugden @mere @pgn-vole @dantrain @bitoiu @dantrain @aaronhaines @haggyj @jhollingworth

[mstade]

I do not mean to imply here that a URN does/should follow the deprecated naming convention (prefixed with urn:{nid}:{nss} etc).

I don't think the leading urn: is deprecated, but if you've got sources I'd happily stand corrected. If you omit the leading urn:, i.e. the scheme, how would you know what the semantics of the rest of the URI is?

URC: is metadata for a resource. Currently, I'm using "headers" to describe this, but URC seems better to describe permanent resource metadata "at rest", whilst "headers" implies to me metadata in the context of a message transfer between a particular sender/receiver.

Headers may be attached to a message, but is just as much "tied" to a resource as the URI or other data. Consider the Link header for instance, would you not think that's part of the resource just because it's represented as a header as opposed to the body? Don't forget that the "stateless" bit in REST just means you don't need additional state to understand a message, not that the data encoded in the message isn't stateful or otherwise "belongs" to a resource.

As far as I know, URC never went anywhere, and URN and URL are basically bundled up in the term URI to make it less confusing to talk about both at the same time. Whether or not a URI can be resolved into an actual location (i.e. it is also a URL) is probably more tied to the scheme semantics than anything. For instance urn: URI will never let you know just by looking at it that you can resolve it to a representation of anything, but a http: URI suggest you can at least to an HTTP GET. Doesn't mean you'll get a representation of any kind of resource, but the scheme at least says you can try.

If we allowed sever nodes to communicate, how would we demarcate namespaces? More importantly, how can this be achieved in a decentralised fashion? This is where the allure of URN's comes back over URL's..

I disagree, you're just overloading URNs at this point. Instead, consider the generic URI syntax which states that the authority component is preceded by //. Thus, you can most certainly have the same paths but different authorities, used to disambiguate. No need to introduce multiple schemes (and indeed overload them with semantics the scheme never supported in the first place) to solve a problem that has already been solved.

[pemrouz]

Thanks for the feedback @mstade! So, what would be the final API you would propose? (definition, declarative and imperative usage example would be appreciated).

I don't think the leading urn: is deprecated, but if you've got sources I'd happily stand corrected. If you omit the leading urn:, i.e. the scheme, how would you know what the semantics of the rest of the URI is?

Sorry, by deprecated I should have said no longer used. More importantly, I was making the point that the leading urn: is not important since just "tweets" fits the definition of "globally unique, persistent identifier".

Headers may be attached to a message, but is just as much "tied" to a resource as the URI or other data. Consider the Link header for instance, would you not think that's part of the resource just because it's represented as a header as opposed to the body?

Agree. But my hesitation is that the problem with using "headers" to describe resource metadata at rest, is not the fields that are permanently fixed (link), but the ones that are not. In the process of sending a resource to two different clients, it is likely that some header fields/values will not be the same. This indicates that they are generally specific to the transfer (i.e. HTTP) rather than strictly the resource, or even it's many possible representations. Thoughts?

For instance urn: URI will never let you know just by looking at it that you can resolve it to a representation of anything

This one is tangential, but see the sentence:

"An individual scheme does not have to be classified as being just one of "name" or "locator". Instances of URIs from any given scheme may have the characteristics of names or locators or both, often depending on the persistence and care in the assignment of identifiers by the naming authority, rather than on any quality of the scheme." [RFC3986]

i.e., within a particular context (ripple) you could know how to resolve that (urn:tweets).

I disagree, you're just overloading URNs at this point.

How exactly do you mean URNs being overloaded here?

Instead, consider the generic URI syntax which states that the authority component is preceded by //

Going back to the original question at the top of this post, could you give some examples of how you are thinking this would look practically? Would everything be prefixed with //, etc?

[mstade]

Sorry, by deprecated I should have said no longer used. More importantly, I was making the point that the leading urn: is not important since just "tweets" fits the definition of "globally unique, persistent identifier".

Potato/tomato I guess. Any string or number fits that definition to be honest; identifiers are only as useful as their context, which I suppose is your point. But depending on environmental context (i.e. ripple) to attach semantics to an identifier is out-of-band galore and prone to break whenever that environment changes. (Because it will.)

To that point, I think there's some confirmation bias going on here:

[...] within a particular context (ripple) you could know how to resolve that (urn:tweets).

I don't think that's what RFC3986 is trying to say. Rather, it's saying that the scheme shouldn't have to declare all possible permutations, and that the naming authority (e.g. in the case of HTTP this would be the domain) can and will have the power to create and maintain new identifiers. The scheme should still define the semantics of the URI, otherwise anyone looking to use these identifiers would have to understand out-of-band information such as environment implementation. The beauty of something like http URIs is that I don't have to understand whether the naming authority is an Apache server, or Node.js, or nginx or whatever, because the common contract here is the scheme, even though the scheme doesn't dictate all identifiers.

Now, if you designed your own scheme called ripple: or whatever, you could dictate all these rules of context as much as you'd like, and anyone pretending to truly understand these URIs would look to the ripple: scheme definition (ideally IANA registered of course) to know how to interpret them. (To be clear: I don't think designing your own scheme is a good idea.)

How exactly do you mean URNs being overloaded here?

Well, the URN scheme explicitly says the identifier are location-independent.

Going back to the original question at the top of this post, could you give some examples of how you are thinking this would look practically? Would everything be prefixed with //, etc?

Yes, I'd say. Anything not prefixed with // should probably belong to some anonymous, global authority. Going schemeless has its advantages too, even if everything goes over HTTP(S). On this note, I'm looking to implement support for naming authority and URI templates in rhumb. I've got most of the work to support the latter done, but busy with other things at the moment.

[pemrouz]

@mstade agree with your comments and looking forward to see the work on rhumb. Still not exactly clear on how you would define/invoke a resource though - could you give a couple of examples? It may be easier to discuss and explain in person what I was thinking re:decentralised resource (:coffee:?)..

[mstade]

Sounds like a splendid idea, I think we're overdue for a tete-a-tete anyhow. :o)

[pemrouz]

More discussion around data-fetching layer in rijs/hypermedia#1. Leaning towards adding first-class support for subresources via dot notation in the next release. This is much easier to decompose, dedupe, manage subscriptions, etc than the GraphQL syntax.