BlurHash is a compact representation of a placeholder for an image (or a frame of video). Currently, in Matrix, clients must display a placeholder image in the message timeline while a piece of media is loading. Some clients, such as Element, simply display an empty space.
While thumbnails exist to combat this to some degree, they still need to be downloaded from a homeserver, which is not instantaneous.
Instead, a BlurHash can be sent inside events, which upon
receipt other clients can render for a pretty media preview while the actual
thumbnail downloads. They also do not contain any " characters, making them
simple to stick inside existing JSON blobs.
To be clear: A BlurHash does not replace a thumbnail - rather it is a placeholder and should be shown before the thumbnail is downloaded. For a familiar messaging user experience, clients are recommended to first render and display a blurhash, then download the thumbnail of the media. Once the thumbnail file is downloaded, display it. Finally, the user can optionally view the full media item by selecting it.
An optional field is added in m.room.message's content.info dictionary
with the key blurhash. It is a BlurHash of the original piece of media.
Clients could then render this using one of the available BlurHash
implementations.
This would be optionally displayed while the thumbnail of the media is loaded in parallel.
Example m.room.message content:
{
"body": "image.png",
"info": {
"size": 149234,
"mimetype": "image/png",
"thumbnail_info": {
"w": 301,
"h": 193,
"mimetype": "image/png",
"size": 72958
},
"w": 301,
"h": 193,
"thumbnail_url": "mxc://example.org/abcdefg",
"blurhash": "JadR*.7kCMdnj"
},
"msgtype": "m.image",
"url": "mxc://example.org/abcde"
}Note that a BlurHash representation is really only applicable to media, and
as such should only be used in conjunction with the following
m.room.message msgtypes:
m.imagem.video
An optional field is added to m.sticker's content.info dictionary with
the key blurhash. Its value is a BlurHash of the sticker media.
Example m.sticker content:
{
"body": "Landing",
"info": {
"h": 200,
"mimetype": "image/png",
"size": 73602,
"thumbnail_info": {
"h": 200,
"mimetype": "image/png",
"size": 73602,
"w": 140
},
"thumbnail_url": "mxc://matrix.org/sHhqkFCvSkFwtmvtETOtKnLP",
"w": 140,
"blurhash": "JadR*.7kCMdnj"
},
"url": "mxc://matrix.org/sHhqkFCvSkFwtmvtETOtKnLP"
}Room avatars having BlurHashes available will be especially useful when viewing a server's Public Rooms directory.
An optional field is added to m.room.avatar's content.info dictionary with the
key blurhash. Its value is a BlurHash of the media that is pointed to by
url.
Example m.room.avatar content:
{
"url": "mxc://matrix.example.com/a59ee02f180677d83d1b57d366127f8e1afdd4ed",
"info": {
"blurhash": "JadR*.7kCMdnj"
}
}Much like room avatars, user avatars can have BlurHashes as well. There is a little more required to implement this, but the outcome of no longer having missing avatars upon opening a room is worthwhile.
An optional field is added to m.room.member's content dictionary with
the key blurhash. Its value is a BlurHash of the media that is pointed
to by avatar_url.
Note that blurhash MUST be omitted if avatar_url is not present.
Example m.room.member event content:
{
"avatar_url": "mxc://example.org/SEsfnsuifSDFSSEF",
"displayname": "Alice Margatroid",
"membership": "join",
"blurhash": "JadR*.7kCMdnj"
}Endpoints that return profile information, and thus MXC URLs to user avatars, are extended to optionally include BlurHashes as well.
GET /_matrix/client/v3/profile/{userId}
has an optional field added with
the key blurhash. Its value is a BlurHash of the media that is pointed to
by avatar_url. blurhash MUST be omitted if avatar_url is not present.
The same applies
to GET /_matrix/client/v3/profile/{userId}/avatar_url
, and to the federation
endpoint GET /_matrix/federation/v1/query/profile
.
GET /_matrix/client/v3/profile/{userId}
{
"avatar_url": "mxc://matrix.org/SDGdghriugerRg",
"displayname": "Alice Margatroid",
"blurhash": "oyp8ky2BWn7VHEL"
}GET /_matrix/federation/v1/query/profile
{
"avatar_url": "mxc://matrix.org/SDGdghriugerRg",
"displayname": "Alice Margatroid",
"blurhash": "oyp8ky2BWn7VHEL"
}GET /_matrix/client/v3/profile/{userId}/avatar_url
{
"avatar_url": "mxc://matrix.org/SDGdghriugerRg",
"blurhash": "oyp8ky2BWn7VHEL"
}Separately, PUT /_matrix/client/v3/profile/{userId}/avatar_url
has an optional field added
to the request body with the key blurhash. Its value is a BlurHash of the media that is pointed to by the value of
the avatar_url field in the same request.
Request:
PUT /_matrix/client/v3/profile/{userId}/avatar_url HTTP/1.1
{
"avatar_url": "mxc://matrix.org/SDGdghriugerRg",
"blurhash": "oyp8ky2BWn7VHEL"
}
Successful response:
200 OK
{}
An optional attribute is added to the OpenGraph data returned by a call
to
GET /_matrix/media/v3/preview_url
called matrix:image:blurhash. The value
of this attribute is the blurhash representation of the media specified
by og:image. Note that we place this value under the matrix namespace as
the OpenGraph protocol does not (yet) have a spec'd field
for BlurHashes.
Note that matrix:image:blurhash MUST be omitted if og:image is not present
in the response.
Example response to GET /_matrix/media/v3/preview_url:
{
"og:title": "Matrix Blog Post",
"og:description": "This is a really cool blog post from matrix.org",
"og:image": "mxc://example.com/ascERGshawAWawugaAcauga",
"og:image:type": "image/png",
"og:image:height": 48,
"og:image:width": 48,
"matrix:image:size": 102400,
"matrix:image:blurhash": "oyp8ky2BWn7VHEL"
}An optional attribute is added to <img> tags in messages:
data-mx-blurhash, where the value of the attribute is the blurhash
representation of the inline image.
This would be optionally displayed while the inline image itself is loaded in parallel.
Example m.room.message.formatted_body:
"formatted_body": This is awesome <img alt=\"flutterjoy\" title=\"flutterjoy\" height=\"32\" src=\"mxc://matrix.example.org/abc\" data-mx-blurhash=\"LEHV6nWB2yk8pyo\" />
BlurHashes are inserted into events by the client, however some clients may not be able to implement the BlurHash library for whatever reason. In this case, it would be nice to allow the media repository to calculate the BlurHash of a piece of media for the client, similar to how thumbnails are calculated by media repositories today.
A new boolean query parameter, generate_blurhash, is added to
POST /_matrix/media/v3/upload
which allows clients to ask the server to generate a BlurHash for them. This
may be useful for clients that are designed to run on very low-power hardware,
or otherwise cannot generate a BlurHash itself.
If set to true, the server SHOULD generate a BlurHash of the uploaded media
and return it as the value of the blurhash key in the response.
If the server cannot generate a BlurHash of the media - perhaps because it is
not a file that a BlurHash can be derived from or it is too expensive to process
a very large piece of media - then the server SHOULD NOT return a blurhash key
in the response. Additionally, if generate_blurhash is not true, then the
server SHOULD NOT return a blurhash key in the response.
Fundamentally, this means that clients SHOULD NOT assume that a server will always
return a BlurHash in the response to /_matrix/media/v3/upload, even if they have
set the generate_blurhash query parameter to true in the request.
An example request from a client that would like the server to generate a blurhash would look like:
POST /_matrix/media/v3/upload?generate_blurhash=true&filename=My+Family+Photo.jpeg HTTP/1.1
Content-Type: image/jpeg
<bytes>
Example response:
{
"content_uri": "mxc://example.com/abcde123",
"blurhash": "LKO2?U%2Tw=w]~RB"
}
We explicitly make this behaviour opt-in as it is assumed that the majority of clients that end up supporting BlurHashes will be capable of generating them locally. Thus, the less load we can put on the homeserver (by not making blurhash generation the default) the better.
Note that media servers will not be able to return a BlurHash string for encrypted media; that must be left to the client.
The server could additionally return the BlurHash string for an image when given an MXC URL. This would be through something like the Media Information API (specified in MSC2380), or similar.
Viewing an image message that is loading:
Once the image loads:
As a sample, this is Element's behaviour prior to this MSC's introduction:
We could include a base64 thumbnail of the image in the event, but blurhash produces much more efficient textual representations.
Older clients would ignore the new blurhash parameter.
Newer clients would only show it if it exists.
Users who have not specified blurhash in their m.room.member event yet may
stand out from users who have while both are loading. This is entirely up to
clients to handle, though a suggestion may be to "fake" a blurhash by
blurring some placeholder image (perhaps something with variation between
users like Element's coloured backgrounds with letters in them, or an
identicon derived from the user's
ID) until the user's actual avatar loads.
Implementations wishing to add this before this MSC is merged can do so with the following:
-
The
blurhashkey in any events, request or response bodies should be replaced withxyz.amorgan.blurhash. -
/_matrix/media/v3/upload's newgenerate_blurhashquery parameter should instead bexyz.amorgan.generate_blurhash. And instead of the keyblurhash, the endpoint should returnxyz.amorgan.blurhash. -
The
data-mx-blurhashattribute in<img>tags should be replaced withdata-xyz-amorgan-blurhash.
BlurHash entries in encrypted events, be it as part of the info property,
or <img> tags, should be encrypted along with the rest of the event
content.
A discussion in #matrix-spec questioned whether massive BlurHashes may be a potential DoS vector for clients. The discussion found that only a maximum of 100 x 100 components can be defined by a BlurHash. This may be in the higher range for low-resource (or unoptimised) clients, and clients are free to refuse to render a BlurHash with a large component count, but it shouldn't be a cause for concern.
Invalid BlurHashes should not be rendered.
BlurHash's algorithm description can be found here, which also includes the full output character set.