Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Major spec changes #288

Merged
merged 2 commits into from
May 18, 2020
Merged

Major spec changes #288

merged 2 commits into from
May 18, 2020

Conversation

dunglas
Copy link
Owner

@dunglas dunglas commented May 12, 2020

This a major spec change intended to fix most remaining limitations. I hope that it can be the last breaking change.

Authorization

This is the main change.

The authorization mechanism has been simplified. It's currently the major pain point for users. The concept of target is difficult to understand. In this proposal, targets are gone. The new mechanism is much simpler:

  • An update can be marked as private by setting the private property while publishing it.
  • In order to receive private updates, subscribers must have a JWT, and a URI template matching the private update's topic must be present in the mercure.subscribe claim of this JWT. It's also possible to use a raw string (it can be a URL or just a random identifier) instead of a URI template.

Subscription events

  • The topic of the subscription event now follow the pattern /.well-known/mercure/subscriptions/{subscriptionID}/{topic} (it was htts://mercure.rocks/subscriptions/{topic}/{subscriptionID}). Using well-known URL for this looks cleaner to me.
  • As the target mechanism is gone, events are now private updates.
  • It is now possible to store user-defined data in the JWT of the subscriber (in mercure.payload), and this user-defined payload is included in the subscription events. It is convenient to store a user ID, an IP, groups and so on.

Event Store and State Reconciliation

The Hub can now be used as an event store (to implement event sourcing for instance) thanks to a simple trick: the event ID -1 is now reserved, and using -1 as value for the Last-Event-ID header or query parameter now hints the hub to send the full history for the subscribed topic(s).

I'm not sure about -1, it could be 0 too, or all, or any other reserved value. Don't hesitate to share your thoughts about this.

Identifiers

It's now possible and recommended to use IRIs everywhere: as topics and as event ids. For convenience, it's also possible to use random string (not IRIs) everywhere.

Encryption

The key set (JWKS) is now exposed through an external URL.

Next steps

  • The hub must be updated accordingly.
  • I also plan to define a tiny REST API in the spec allowing to fetch the list of active subscriptions.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
@dmouse
Copy link

dmouse commented May 13, 2020

About mercure.payload, can it be used as a sub-topic?

For example:
This messages is delivered only for subscribed galgo dogs

htts://mercure.rocks/subscriptions/vendor/dogs
{
breed: "galgo",
}

Is it correct?

@dunglas dunglas mentioned this pull request May 14, 2020
3 tasks
@dunglas
Copy link
Owner Author

dunglas commented May 18, 2020

@dmouse Sorry I don't understand exactly what you mean. The payload is a random data structure you add to the subscriber's JWT. Then, this data structure is attached to the subscriber, and included in subscription events related for this subscriber.

@dunglas dunglas merged commit 6c12fd7 into master May 18, 2020
@dunglas dunglas deleted the revamp-spec branch May 18, 2020 08:04
Copy link
Contributor

@sroze sroze left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise, very nice 👌

The hub **MAY** discard some messages for operational reasons. The subscriber **MUST NOT** assume
that no update will be lost, and **MUST** re-fetch the original topic to ensure this (for instance,
after a long disconnection time).
The reserved value `-1` can be used to hint the hub to send all updates it has for the subscribed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is confusing to me. I'd have expected -1 to actually be used as a relative value to only return the last event. I'd go for a much more explicit all constant.

@a-menshchikov
Copy link

I believe that the rejection of the mechanism of targets was the wrong decision.

Suppose we have a single page application that contains two thousand entries. Even entries owned by userA, and odd entries owned by userB. An entry have properties "id", "status" and "owner". (It's important that an application is SPA: user get an authorization token for Mercure only once, when start an application.)
I want to subscribe users for status updates of their entries. But without the targets mechanism it seems impossible: user must subscribe for concrete topic if we want to make it private (e.g. "https://example.com/entry/123/status"), but if user own a thousand of entries authorization token will have extremely large size and it will be unusable both in cookie or in authorization header while connecting to Mercure.

@dunglas could you describe an example of solution for my example case, when we need to publish status updates in SPA for users with a lot of entries and we can't subscribe it for concrete entries by their enumeration?

@dunglas
Copy link
Owner Author

dunglas commented May 31, 2020

@a-menshchikov you can use URI templates for this use case: https://example.com/entry/{id}/status. If you want user A to receive some updates, and user B receive some others, you can leverage the alternate topic feature:

POST /.well-known/mercure

topic=https://example.com/entry/1/status&topic=https://example.com/entry/forUserA/1/status&private=true

POST /.well-known/mercure

topic=https://example.com/entry/2/status&topic=https://example.com/entry/forUserB/2/status&private=true

Then you can set https://example.com/entry/forUserA/{id}/status in the mercure.subscribe claim of the JWT of user A, and https://example.com/entry/forUserB/{id}/status for user B.

What do you think?

@a-menshchikov
Copy link

@dunglas it seems less clear than it was with targets mechanism, especially if we have tens of different entry classes in our application. Previously, we had to generate JWT for user with one target for himself (user/{username}) and set appropriate target on publication. Now we should generate JWT with enumeration of all kind of entries in topic.

As result we have more complicated code of JWT generation and we cannot segregate JWT generator and core application because it should know about entry classes.

@a-menshchikov
Copy link

Also this approach requires me to solve an issue how to pass the username to client (that need for a subscription url entry/forUserA/{id}/status).

With targets mechanism client doesn't need to know anything about private subscription principle. It basically get JWT from backend that contains information about allowed targets, and then retrieve from hub granted messages. It's much more simpler than need to make JWT with all private topics and pass the username to client.
Finally, it seems extremely hard to implement subscription simultaneously for certain users and user groups.

@dunglas
Copy link
Owner Author

dunglas commented Jun 10, 2020

Previously, we had to generate JWT for user with one target for himself (user/{username}) and set appropriate target on publication. Now we should generate JWT with enumeration of all kind of entries in topic.

This can be solved using URI templates.
The JWT can contain something like mercure.subscribe: ["https://example.com/userA{/any*}"]. Then you can use alternate IRIs just as you were using targets before:

POST /.well-known/mercure

topic=https://example.com/foo/123&topic=https://example.com/userA/foo/123&private=on

Also this approach requires me to solve an issue how to pass the username to client

No because it's just an alternate IRI. Client can still refer to the canonical IRI which doesn't need to contain a reference to the user name. It solves all problems you describe. WDYT?

Regarding the previous example, the client can still subscribe to https://example.com/foo/123 or even https://example.com/foo/{id}.

@a-menshchikov
Copy link

@dunglas this solution works, thank you!

@bats-dm
Copy link

bats-dm commented Sep 29, 2020

@dunglas So if I want to publish the same update for the several users I need to use multiple topics, right?

POST /.well-known/mercure

topic=https://example.com/userA/foo/123&topic=https://example.com/userB/foo/123&topic=https://example.com/user{N}/foo/123&private=on

For example, we have the chatroom in our chat. This chatroom contains 1000 users at the moment. Will it be 1000 topics in POST request with just different users?

@dunglas
Copy link
Owner Author

dunglas commented Sep 29, 2020

For a chat room there is an easier way:

  • messages can have a topic following a pattern similar to https://example.net/the-chat-room/<id> (example: POST topic=https://example.net/thé-chat-room/22)
  • users can bear a JWT containing the template selector https://example.net/thé-chat-room/{id}

This way, you every messages can have a single topic, and all authorized users can receive them.

@bats-dm
Copy link

bats-dm commented Sep 30, 2020

@dunglas I thought about this way, but if we add https://example.net/the-chat-room/{id} to JWT all users have access to all chat rooms and can read messages. But for security reasons they should have access only for rooms they are in. The only way I see is to update the user's JWT every time they in or out a chat room. But the problem that only admins can assign or unassign the user to the room.

So I consider the way I mentioned in the previous post but I want to make sure that I understand everything correctly because I used targets earlier and I want to update Mercure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants