MSC1951: Custom sticker packs and emoji (mk II) #1951
Conversation
turt2live
added
proposal-in-review
proposal
turt2live
changed the title
|
|
||
| The sharable URL for a pack could leak information about who created it or what it contains, potentially | ||
| subjecting the user to personal risk. It is expected that implementations handle this appropriately | ||
| where possible, such as by giving the creator the option to create private packs with randomly generated |
There was a problem hiding this comment.
Randomly generating a room alias is all well and good on matrix.org, but is kind of pointless on a one user HS? I am not sure I can see a way out of that though.
There was a problem hiding this comment.
For single user homeservers it sounds like it would get into MSC1228 territory.
| ```html | ||
| <img src="mxc://example.org/media_id" width="16" height="16" | ||
| alt="This is the item's description" | ||
| data-mx-pack="https://matrix.to/#/%23_stickerpack_sample1%3Aexample.org" /> |
There was a problem hiding this comment.
Are you intending to use the existence of data-mx-pack to differentiate this as a specific type of inline image that clients might want to render in an emoji like way?
There was a problem hiding this comment.
that would be the plan, yup. I apparently didn't say that though - will add.
|
It's not mentioned in the proposal because it doesn't directly apply, but a paid sticker pack would probably specify a publicly reachable page for a sharable URL (eg: In the case of a unique room, the room would probably be open to the public but set up such that the provider has a bot which has made it so the user can't do anything in the room, including invite others. Once the first user joins, the bot would make the room invite only to prevent the user from sharing the link to friends, giving them a free sticker pack. In future, the The third option for paid sticker packs (which isn't as great) is to just treat the stickers as not belonging to a pack, and therefore not needing to provide a URL. |
|
|
||
| There are two kinds of special events which appear in a pack's dedicated room: `m.pack.metadata` for | ||
| the pack's metadata and zero or more `m.pack.item` for the stickers/emoji themselves. Both event types | ||
| are state events. The presence of a `m.pack.metadata` event indicates that it is a pack room. |
There was a problem hiding this comment.
Doing something like https://github.com/matrix-org/matrix-doc/issues/423 would be too big of a can of worms I suppose? Would be nice to have one way of identifying a non-IM room once such a thing lands ...
There was a problem hiding this comment.
Also, clients will need to be able to switch a normal room to a pack room with this approach, as they can't make any assumptions when the m.pack.metadata event will be added.
Just an idea, but making this information part of the createRoom call / m.room.create event could make it more straighforward to deal with non-IM rooms by not having to treat them as IM rooms first.
There was a problem hiding this comment.
Fixing #423 would be out of scope here, but adding a flag to the create event is easily doable. It'd also probably fix the concern of moderators taking over rooms as pack rooms.
|
would love to see this get merged |
|
Anything blocking this from being added to matrix? Is it already wip? Can anyone give a status update? |
|
I'm curious as well, our community has been requesting this constantly. |
|
It's currently on my personal backburner. There's a partial implementation in the form of sticker packs (https://github.com/turt2live/matrix-sticker-manager) though it is far from perfect or ideal. Hoping to push this forward once some of the larger MSCs finish landing. |
|
Thanks for the update. Just another ping to say the community I'm a part of would also love this. |
This comment has been minimized.
This comment has been minimized.
| @@ -0,0 +1,273 @@ | |||
| # Custom emoji and sticker packs in Matrix | |||
There was a problem hiding this comment.
@steef435 said (#1951 (comment)):
Complete noob here. I wonder how privacy-friendly this approach is?
If I understand correctly, this proposal does not do any E2EE. It's all state events, which don't get encrypted IIRC. It would leak both sticker content as well as who is using which sticker packs (to those who are in the sticker room / admins of any HS of which at least one user uses the pack). I feel this makes creating "personal" sticker packs (pictures of company property, personal pictures of a group of friends, et cetera: private packs for use in small communities) sketchy.
If I understand the spec correctly, encrypted files are only encrypted once, and then the decryption key is sent encrypted to all recipients? Would it be possible to use a similar system for sticker packs? That would at least save them from compromised servers.
The sticker pack room neatly announcing everyone who is using the pack looks like valuable metadata to me in the case of community-specific sticker packs. A different approach I can think of would be sticker packs as files (or maybe one index file with metadata an thumbnails linking to individual sticker files). Makes it harder to come up with a neat update mechanism, maybe there would be none. But it would be easily encryptable. I believe Signal does something like this.
What do you think? Is there any way to make this more privacy-friendly?
(sorry, we use threads to do this sort of thing to track replies)
There was a problem hiding this comment.
Indeed none of this is encrypted, by design. Users don't have to join the room themselves either: they can rely on an integration manager which would join for them (and everyone else using that instance) and therefore hide their identity. This is the preferred approach.
In future, Matrix will support peeking over federation which means you won't even have to join the room.
As for the files themselves: By nature of this being shared information it's not reasonable to add encryption, as it would impede other people's ability to use the stickers (and you'd risk clients forwarding the encrypted info, which might disclose some metadata about the creator). There's nothing stopping this proposal from getting encryption support in the future, though it is intentionally out of scope for this iteration.
In general, this feature is meant for sharing sticker packs, not defining how they must be stored. A privacy-focused client would likely support a feature for encrypted sticker packs that didn't use this MSC, which would be completely reasonable. That client would potentially have some sort of sharing mechanism, though encryption and sharing do not mesh well by design.
To be clear: the intention is that if someone wants to privately share stickers, they do so through some other mechanism. This MSC is intentionally designed and written to support the more common case of publicly shareable information, similar to what you'd stick on a billboard.
There was a problem hiding this comment.
I see, thanks! I hope there will be room for secret sticker in the spec in the future.
Small reply:
As for the files themselves: By nature of this being shared information it's not reasonable to add encryption, as it would impede other people's ability to use the stickers (and you'd risk clients forwarding the encrypted info, which might disclose some metadata about the creator).
The way I think about it is that once you'd receive a sticker, you'd get access to the pack uri and the encryption key through the sticker event. So they would be shareable enough but still confined. I'm not too sure about what you mean by creator metadata, it's fine if a keypair is created and used solely for one pack right?
I now get this is not the scope though. I'll read up on the integration manager concept, I thought that was used to add "things" to rooms, not to clients. Anyway, thanks for your response!
There was a problem hiding this comment.
I think it is best to implement encryption here. I see a number of benifits:
- Not implementing encryption upfront will be a privacy hole. People will find it fun to make personal sticker packs among groups of friends. It will likely not be obvious that they are revealing these to the homeservers even if the chat that they are used in is e2ee.
- If unencrypted sitckers work it will slow adoption of encrypted stickers as they have less compatibility. Sticker pack makers will be incentivized to release new packs unencrypted as they will have wider support.
- IIUC just requesting the image is enough to let the homeserver know what the sticker is. So fetching a "Congratulations on the baby boy" sticker will provide a lot of info to the homeserver (hello ads). (This may not be easy/possible to solve for popular sticker packs other than reuploading the stickers on every use but it would be nice to avoid it for rare or private sticker packs. Without encryption the homeserver can just run OCR on the image and learn from it.
- Encrypted sticker packs are a complete superset of unencrypted sticker packs, public packs just make the key available to everyone. It would be unfortunate to have 2 specs and implementations for the same thing. We may as well make the single spec encrypted and remove the need for the unencrypted spec.
TL;DR I think we should encrypte all sticker packs (and stickers). This means that we have one implementation that is unsurprising with regards to privacy.
| for integration with their autocomplete mechanisms and for general performance. | ||
|
|
||
|
|
||
| ## Security considerations |
There was a problem hiding this comment.
As I understand it, users who want to use a sticker pack will have to join the "virtual" room of that pack.
They will therefore be added to the room's membership list, which is visible to everbody (or only other members for private packs). It would even be possible to see the past users of a pack by looking at the join/leave events of that room.
This might be a privacy issue.
There was a problem hiding this comment.
Ideally we get peeking over federation before then.
|
There doesn't seem to be any activity on this for quite a while. What's blocking this from being merged? Is it not being worked on anymore? |
|
It's just not a priority on my side right now. Generally speaking, #matrix-spec:matrix.org is a better place to be asking for status updates. |
| interaction to a future proposal. Clients are likely to want to handle custom emoji themselves anyhow | ||
| for integration with their autocomplete mechanisms and for general performance. | ||
|
|
||
|
|
There was a problem hiding this comment.
Can I use org.matrix.msc1951. as a namespace for the event types, when I play around with implementing this? Or do you prefer a different one?
There was a problem hiding this comment.
Stickers will not cause any backwards incompatible change, so directly testing on m. seems viable in case of this specific proposal.
There was a problem hiding this comment.
m. is an illegal namespace to use. This MSC does not currently specify a namespace.
|
XMPP accepted stickers specification in their protocol. Maybe will be useful for some inspiration? |
| Sharing can also be done through distributing plain `http` or `https` URLs which are able to point to the | ||
| room in which the pack is defined. The intent is that 3rd party services can create sticker packs and | ||
| distribute user friendly or SEO optimized URLs, such as `https://packs.example.org/1234/official-sheltie-pack`. | ||
| When approached with `Accept: application/json` or have `.json` appended to the end of the URL, the |
There was a problem hiding this comment.
How does the client know which to do? Do you need to try both? I think that a single method (probably Accept) should be used.
If we really want the alternative URL we may allow returning HTML with <link rel=org.matrix.stickerpack href=...>. This way the client still just makes a single HTTP request and can figure out what to do based on the Content-Type of the response. However I would try to get away without this method first, and we can add it later if required.
| ## Proposal | ||
|
|
||
| Custom emoji and sticker packs are the same thing technically, just handled differently in the client. | ||
| Stickers in this proposal are the same `m.sticker`s found in the current Client-Server API specification. |
There was a problem hiding this comment.
It would be good to link for the spec for m.sticker.
| The `description` is required and should be a human representation of the emoji or sticker. The description | ||
| should be a single short sentence, but has no length limit. | ||
|
|
||
| The `uri` is required and is a Matrix Content URI to the represented emoji or sticker. |
There was a problem hiding this comment.
This is an existing problem with m.sticker and m.image but it would be nice if there was support for multiple qualities and formats to support different client technologies and resources.
| The `description` is required and should be a human representation of the emoji or sticker. The description | ||
| should be a single short sentence, but has no length limit. | ||
|
|
||
| The `uri` is required and is a Matrix Content URI to the represented emoji or sticker. |
There was a problem hiding this comment.
Is a client expected to re-thumbnail images every time? Or does it make sense to include a common thumbnail here?
| @@ -0,0 +1,273 @@ | |||
| # Custom emoji and sticker packs in Matrix | |||
There was a problem hiding this comment.
I think it is best to implement encryption here. I see a number of benifits:
- Not implementing encryption upfront will be a privacy hole. People will find it fun to make personal sticker packs among groups of friends. It will likely not be obvious that they are revealing these to the homeservers even if the chat that they are used in is e2ee.
- If unencrypted sitckers work it will slow adoption of encrypted stickers as they have less compatibility. Sticker pack makers will be incentivized to release new packs unencrypted as they will have wider support.
- IIUC just requesting the image is enough to let the homeserver know what the sticker is. So fetching a "Congratulations on the baby boy" sticker will provide a lot of info to the homeserver (hello ads). (This may not be easy/possible to solve for popular sticker packs other than reuploading the stickers on every use but it would be nice to avoid it for rare or private sticker packs. Without encryption the homeserver can just run OCR on the image and learn from it.
- Encrypted sticker packs are a complete superset of unencrypted sticker packs, public packs just make the key available to everyone. It would be unfortunate to have 2 specs and implementations for the same thing. We may as well make the single spec encrypted and remove the need for the unencrypted spec.
TL;DR I think we should encrypte all sticker packs (and stickers). This means that we have one implementation that is unsurprising with regards to privacy.
turt2live
added
the
needs-implementation
|
What is blocking this merge request? |
|
A re-review from me, the author, to compare its feasibility against other MSCs, then a proper implementation, then spec core team review. |
|
we have a lot of people who would really really like this feature, and its absence is one of the biggest problems with matrix for us |
This comment has been minimized.
This comment has been minimized.
|
Please refrain from offtopic discussion and imagery. |
|
@turt2live Thanks a lot for the PR |
|
The MSC can be read from the files tab. If you have comments or concerns, please leave them inline on that view. |
| `matrix.to` (in future `mx://`) URI. | ||
|
|
||
| The `license` is required and must be a valid SPDX identifier. This applies to all the items contained | ||
| within the pack. |
There was a problem hiding this comment.
This is not distributing software, and often, the licensing for non-software has different concerns. As such, I find it a little strange that we're tying licensing to a system specifically for software (it's in the name!)
While I think it's ok to just use the SPDX identifiers to facilitate CC usage, I think this needs a little more attention in the direction of custom licensing, which doesn't seem to be possible right now.
This comment has been minimized.
This comment has been minimized.
|
Looking through the proposal shouldn't there also be a way to define per-user stickers/emoji? Also what about animated stickers/emoji? |
turt2live
added
the
blocked
| good opportunity to render the sticker pack for viewing by a human. | ||
|
|
||
|
|
||
| #### Recommending which packs to use in a room |
There was a problem hiding this comment.
we now have Spaces, which can feature children. If packs are children, they can be featured the same way.
| Because access control is handled by Matrix itself, applications should not omit the field if the pack | ||
| is private and instead send it anyways. |
There was a problem hiding this comment.
I couldn't actually find any explanation of how the access control is supposed to work - could you clarify?
| data-mx-pack="https://matrix.to/#/%23_stickerpack_sample1%3Aexample.org" /> | ||
| ``` | ||
|
|
||
| The `src` is the `mxc://` URI of the item, and the `alt` is the description of the item. The `width` |
There was a problem hiding this comment.
under MSC3911 it would need to be the URI of a copy of the item.
| Implementations should use the event timeline as a sort of comment section for the pack. | ||
|
|
||
|
|
||
| #### Sharing packs with other users |
There was a problem hiding this comment.
This sort of mechanism was designed in an era where it might be theoretically useful. It turned out not to be useful - it should be removed from this MSC.
turt2live
added
obsolete
Rendered
Obsoletes MSC1256
Changes from MSC1256:
This is proposed with a community hat on: