Explain manual and automated key backup better

[aaronraimist]

Lots of people tend to think manual key backup is something like GPG, ie if you export the key as soon as you create your account you'll be able to decrypt all future messages that are sent after you exported the keys.

[aaronraimist]

It also needs to be more clear that key backups to the server are stored on your homeserver. Not some central matrix.org server.

[drathir]

Agree clarify of remote storage in use would be nice in description, also taking on mind if that some kind of private kinda keys would be nice as well get possibility of gpg/passphrase/one time paste style send to email address protection type i guess (theoretically speaking only, bc not checked/searched for type of data send by this feature).

[matthijskooijman]

I just tried setting up automatic key backup and found that there's a lot of stuff that's not really well-explained through the UI. Here's some questions that crossed my mind:

• What is stored on the server exactly? A backup of my room keys, I suppose. Encrypted using a recovery key? Or my passphrase? Or both?
• Where is the backup decrypted? I assume in the browser, so the server never sees the actual keys? Of course, a malicious server could still easily inject malicious javascript to sniff the passphrase and keys, of course...
• How do the recovery key and the passphrase correspond to each other? It seems these are basically equivalent in how they can be used? Or does the passphrase normally encrypt the recovery key?
• How does key backup work with multiple devices? Normally, multiple devices seem to have their own E2E keys I believe? Does key backup allow sharing a single set of keys among devices? Or will they just have their own parallel backups? How does this interact with the key sharing feature?
• Does the recovery key ever change? I've tried a some things with different browsers (also coming back to existing browsers) and found that I got different keys.
• As long as your browser (or other device using key backup) remains logged in, you can always access the original keys, and also setup new backups (even if you forgot your passphrase or recovery keys), right?
• When I enter a passphrase or recovery key, is it stored in the browser? I guess it should (or perhaps an underlying backup encryption key) since it must be able to add new keys to the backup later?
• What does it mean for a backup to (not) have a signature from a device?
• How are backup versions handled? Does any change to the backup (content / keys / etc) result in a new version?

I don't think all of these questions should be answered in the interface directly, but perhaps the UI could be changed to support a more useful mental model of what goes on in the background, and perhaps also link to more complete documentation (I just found https://about.riot.im/help#end-to-end-encryption which has some info already).

[lampholder]

Agreed - we're trying to strike a balance between informative and overwhelming, and to provide enough information to reassure encryption/privacy novices, experts and everyone in between.

This list of questions is very useful - we can take another look at the wording/UX with this in mind and see what we can change to answer more of these questions as you work your way through setup.

[turt2live]

This is becoming more prevalent thanks to recent events. There's numerous reports of people taking their exported keys from months/years ago and trying to import them, expecting that to work. We should fix the messaging so that in N years we don't have to tell people that export !== backup.

[cyphar]

I will admit when I first turned on key backups I went to read the MSC to make sure I understood how the system actually worked -- which is far from ideal if we want the security of the system to be self-apparent (as an aside I think the design is really brilliant). I think the simplest way to improve people's understanding of how key backups work is to apply a couple of helpful tips (which might help some of @matthijskooijman's questions):

• Add a (probably red) warning to the key export feature explaining that in encrypted rooms the session keys will roll over quite regularly and thus exported keys will not be useful for decrypting future messages soon after you've done the export. This is the biggest mistake I've seen and it's a shame we didn't have better text on this before (I know I've made this mistake before -- and only realised when I noticed the key exports were getting larger each time I did an export).

  • There should also be a button for using key backup on the key export page to hopefully explain that key export requires more work on the users' side. Most of the text on key export could probably be dropped to make it more likely people will read it.

• Change the key backup setup flow, so that you first get the recovery key and then are asked for the passphrase (with some text on the passphrase prompt saying that it's optional -- but recommended -- that the recovery key be encrypted and saved to the homeserver so that the user doesn't have to keep track of it). This might help users understand that the recovery key is the primary recovery system and the passphrase is just there to not require you to keep track of the recovery key (the current text is "We'll store an encrypted copy of your keys on our server. Protect your backup with a passphrase to keep it secure." which is quite misleadingly-written -- arguably it's more secure to not set a passphrase since then the private key never hits the server).

• I would suggest having a (skippable) "getting started" screen for enabling e2e encryption which might explain some of the questions that @matthijskooijman mentioned above. Since the plan is to make all private rooms e2e-only we should probably work on the onboarding flow now rather than later.

  • I think there is a fairly good "intuitive" way of explaining how key backups work (you could even do it graphically to help get the point across) -- padlocks and keys. When you set up key backups, you create a new key and a large number of corresponding padlocks. You keep this key safe, and whenever you're chatting with someone and you get a new session key you put it inside a safe box and lock it with a padlock and send it to the homeserver -- you only need the recovery key when you need to do a recovery. I bet that a lot of confusion could be avoided with some picture-based explanations (something like the Keybase art style would be great IMHO).

[jryans]

Perhaps we could restructure some of our in-app messaging to show first the simplified explanation for everyone, but then add a disclosure something like:

More details

Advanced encryption explanation details go here.

so that we can still offer more detail for those who want without overwhelming those you don't.

[cyphar]

That would be a good stop-gap to at least provide some sort of information to those who should be aware of things, but I think that all the E2E messaging needs to be made layperson-friendly if we want people to turn it on (or make it the default) without being surprised when the last 12 months of chat history with their loved ones has gone up in smoke (I'm a technical user and this happened to me, but it's a painful mistake to make).

I'm aware (and really grateful) that the technical side of this is being worked on very hard (and the results speak for themselves -- the new e2e features are really top-notch), but making a good on-boarding experience with encryption (or even Matrix in general) would really help users avoid some pitfalls.

I'm fairly convinced the best shot of making it reasonable is to do it graphically (I'm really not an artist but I could knock up some mock-ups to hopefully show why graphical explanations would help). Unfortunately you'd need a designer to actually make them usable -- and they are painfully hard to find within free software communities (despite how desperately we need them in most free software projects).

[35609902357]

@cyphar

it's more secure to not set a passphrase since then the private key never hits the server

Is this a thing embedded into the client (such as Riot) or does this happen at the protocol level (Matrix)?
Is it possible to delete the backup from the home server once the passphrase has been inserted by removing the passphrase or are there any methods to do it?

[cyphar]

Is this a thing embedded into the client (such as Riot) or does this happen at the protocol level (Matrix)?

The protocol defines how key backups work. Basically, your "recovery key" is the private half of an X25519 keypair. All new session keys get encrypted by clients using the public key (so you only need the private half when you first import your keys). Optionally, you can symmetrically encrypt the X25519 private key with a passphrase, and then that encrypted private key gets stored on the server. But since the private key is generated on your client, if you don't set a passphrase then the server never sees the private key.

Is it possible to delete the backup from the home server once the passphrase has been inserted by removing the passphrase or are there any methods to do it?

There's a "delete backup" button in the key backup settings. I don't think you can currently delete only the passphrase (while keeping the backup), but if you delete the backup and then create a new one it will contain all of your keys (and you can not add a passphrase).

[35609902357]

@cyphar I'm having trouble understanding how it works. Whether I encrypt the backup with a passphrase or skip the passphrase and go directly for the recovery key, the backup status shows Backup key stored: not stored in both cases, but I figure it should say it's stored if I input a passphrase, thus saving it on the homeserver, why is it not the case?

Moreover when restoring a backup protected with a passphrase I get the passphrase prompt, but if I can't remember it I can skip to the recovery key without providing the passphrase, but then where does it retrieve the backup from? If it's on the device's internal storage then skipping the passphrase and inserting the recovery key on another device added for the first time should not recover any key, is this correct?

[cyphar]

@35609902357

but I figure it should say it's stored if I input a passphrase, thus saving it on the homeserver, why is it not the case?

I think it should say that too, that might be a bug, I'm not sure.

Moreover when restoring a backup protected with a passphrase I get the passphrase prompt, but if I can't remember it I can skip to the recovery key without providing the passphrase, but then where does it retrieve the backup from?

The actual session keys (the keys you are backing up) are always stored on your homeserver, encrypted using a asymmetric Curve25519 keypair. If you enter the "recovery key" you are actually entering the private key for a Curve25519 keypair (the server keeps the public half, and gives it to your devices -- so they can add entries to your key backup without storing or prompting for the private key). However if you add a passphrase then your homeserver also stores a copy of the Curve25519 private key (the "recovery key"), encrypted with the passphrase you chose.

From my understanding this design has changed somewhat since the introduction of SSSS (which is how cross-signing has been implemented), but the broad strokes should be the same.

[35609902357]

@cyphar
So inserting a passphrase and saving the private key on the homeserver simply means adds the convenience of choosing the passphrase to restore backups instead of saving the random recovery key?

And that IF you choose a strong passphrase the history is safe, but if the passphrase is weak then you might potentially compromise the security of all your messages?

So if I lose access to all my devices, and I only have my passphrases, I add another device without any trace of previous local activities/backups, would I be able to recover all my encrypted history by only providing the recovery key? Would the only difference be inserting the recovery key by myself instead of inserting the passphrase to decrypt it and letting the homeserver insert it for me?

[cyphar]

@35609902357

So inserting a passphrase and saving the private key on the homeserver simply means adds the convenience of choosing the passphrase to restore backups instead of saving the random recovery key? And that IF you choose a strong passphrase the history is safe, but if the passphrase is weak then you might potentially compromise the security of all your messages?

Yes, though in order to exploit this you'd need to either be a homeserver owner or have the users' password to get access to the key backup.

So if I lose access to all my devices, and I only have my passphrases, I add another device without any trace of previous local activities/backups, would I be able to recover all my encrypted history by only providing the recovery key?

Yes. This is what I do, and I have lost my devices before (both before and after the introduction of key backup -- it was much easier to deal with when key backup was added).

Would the only difference be inserting the recovery key by myself instead of inserting the passphrase to decrypt it and letting the homeserver insert it for me?

Yes, though the server doesn't do the decryption of the recovery key or key backup for you (for obvious reasons) -- it's done client-side. Aside from that, that is correct.

[35609902357]

@cyphar
Then it sounds like a silly feature to have, not only it's a waste of coding effort, but it can only endanger the security of the backup because a passphrase secure at least as the recovery key will most likely have to be stored in a password manager (or other means) anyway, and the average Joe will certainly choose a weak passphrase. So the whole thing sounds entirely like a detriment and should not be explained better, but totally removed. Or are there any upsides I'm not seeing to having it?

[ara4n]

So the whole thing sounds entirely like a detriment and should not be explained better, but totally removed. Or are there any upsides I'm not seeing to having it?

It's hard to guess what you're not seeing.

The pros are:

• You can recover all your message keys if you lose all your devices (particularly easy if you only have one device).
• You don't have to rely on manually exporting your message keys.

The cons are:

• If you choose a weak passphrase, someone with access to the backup (a server admin, or someone who has compromised your account password) can get all your message keys.

It's not clear why you think that storing a recovery key or recovery passphrase in a password manager is a bad thing.

[cyphar]

@ara4n To be fair, they're arguing that recovery passphrases are a "bad feature" because if the user could pick a strong passphrase they're probably using a password manager and thus could just store the recovery key. If they can't store a recovery key, then their recovery passphrase is probably weak and we shouldn't encourage them to use passwords if they're likely to be weak. Not to mention that most users (sadly) do not use password managers.

My view is that it's a trade-off which is entirely up to the user, and the pain of losing messages with your loved ones is much more likely to disenfranchise users than any system which permits weak passwords. (Not to mention losing your messages was basically guaranteed with the old key backup system, because you had to constantly keep backing it up -- something that even more technical users like myself managed to misunderstand.)

Personally I think we could do more to make authentication less vulnerable to bad passwords (such as having 2FA support 😉), but this is perhaps an example where explaining to users that the password is optional (and should be very strong) but that it requires more work on their side is probably a good idea.

[ara4n]

oh, okay. yeah, we could forbid recovery passphrases and just mandate recovery keys.

[35609902357]

@ara4n I apologize if I came across as dismissive, it was not my intention. @cyphar made an excellent job explaining my concern and providing the reason I was not able to see, that there are users who care more about preserving message history than security. Then yes, it makes sense passphrases should stay, but should be explained better to make as clear as possible the risks stemming from a weak passphrase, and above all the fact that providing the passphrase itself is not mandatory, nor added security and that it means the recovery key will touch the server if provided. It should also be the other way around, the recovery key should be provided at first with a secondary toggle to enable passphrase backup.

[cyphar]

It should also be the other way around, the recovery key should be provided at first with a secondary toggle to enable passphrase backup.

I agree with this suggestion -- I struggled to understand what was going on with the key backup setup flow until I looked up the MSC. If it was explained as "here is the important secret, and if you can't remember it safely you can store it on the server by encrypting it with a passphrase" it would be much more approachable -- right now it looks more like the recovery key is a randomly-generated passphrase because you have to click away from a password dialogue to get it.

[LinAGKar]

It sounds too complicated for normies, probably that feature can't be implemented in a way that's both safe and easy, then just forbidding the users to use the account password for key backup should at least avoid disasters in case of password leaks. Then when cross-signing will be properly implemented it will further reduce chances of losing data.

The same thing I suggested is already used by Firefox sync: https://hacks.mozilla.org/2018/11/firefox-sync-privacy/

[LinAGKar]

And deriving both the account key and the recovery key from the user entered password would allow completely getting rid of recovery key management and device verification, since it could establish trust using the derived recovery key just logging in, which would significantly reduce friction for new users. Having to explicitly verify devices to establish trust could then be an advanced option most users wouldn't need.

Add TOFU to that, and most people would never to bother with device verification.

[t3chguy]

@LinAGKar it would also be incompatible with LDAP auth and SSO/SAML

[LinAGKar]

Apparently they are planning to allow using a single password for both: https://github.com/vector-im/roadmap/projects/1#card-48806427

[c33s]

do we really want our private keys on a server only protected by a password where the eu is planning to install backdoors in our end to end encryption?

• https://www.eff.org/deeplinks/2020/10/orders-top-eus-timetable-dismantling-end-end-encryption
• https://www.androidpolice.com/2020/11/09/the-eu-may-be-looking-into-breaking-end-to-end-encrypted-chats/

i would really love to see good local backups or support to backup to an ssh or webdav server.

for crypto currency we use hardware wallets and say "not your keys not your coins" and in communication it is ok to auto backup the keys to the cloud without a different option?

[t3chguy]

You are not at all forced to backup your keys to the cloud. You have manual export functions inside Element. Keep in mind that due to Perfect Forward Secrecy - https://en.wikipedia.org/wiki/Forward_secrecy - you will need to backup very frequently if you do not want to lose your messages.

Element as a thing which runs inside a web environment cannot make SSH connections so that's simply out of the question.

[LinAGKar]

What about automatic backup to a local file (which can then be synced/backed up some other way)?

Also, WebDAV ought to be simple for a web app, and SSH (presumably SFTP) should be doable in the desktop app, albeit not in a web browser.

[c33s]

well i felt really forced to use cloud backup with this nag screen comming up every time i started riot (where i asked for clear description in the app element-hq/element-web#13685).
i would love to have a local auto-backup which saves me from loosing access to my message history (*) because after my harddisk ran full i simply lost my keys/access to all my message history element-hq/element-web#15788 (i assume that it happend because of key rotation but why keys are getting rotated if no space is left on the device? (freeing up space and restarting the app has not brought back my messages/keys))

*: i really miss a feature to permanent decrypt a message locally. i someone is on my system where they keys reside the encrypted db wont really help

[t3chguy]

What about automatic backup to a local file (which can then be synced/backed up some other way)?

Web apps can't automatically write to your files, you can only trigger file downloads upon an explicit user action.

why keys are getting rotated if no space is left on the device

Web apps can't check how much space is left on your device.

[c33s]

does both of your comments also apply to the desktop app (i only use the desktop and there is no issue tracker for the desktop app)?

Web apps can't automatically write to your files, you can only trigger file downloads upon an explicit user action.

maybe trigger onclick & onfocus & onmessage ,... with enough trigger which trigger a "cron manager" which backups the keys around every x days? of course it still can happen that no backup occurs but it is still better than no backup (or remote backup)

Web apps can't check how much space is left on your device.

oh. that's really bad but then there really should be a much better, for not tech people understandable info what no backup means and what can happen if no manual backup is made.
as it's a webapp there is also an option to allow api acces where i can trigger the download/an interaction by a local cron job.

[t3chguy]

does both of your comments also apply to the desktop app (i only use the desktop and there is no issue tracker for the desktop app)?

In the desktop app these limitations are not present but require much more additional work.

maybe trigger onclick & onfocus & onmessage ,... with enough trigger which trigger a "cron manager" which backups the keys around every x days? of course it still can happen that no backup occurs but it is still better than no backup (or remote backup)

The user experience this creates will be awful, the app randomly downloading files and your Downloads folder having a lot of random looking files.

[LinAGKar]

maybe trigger onclick & onfocus & onmessage ,... with enough trigger which trigger a "cron manager" which backups the keys around every x days? of course it still can happen that no backup occurs but it is still better than no backup (or remote backup)

I'm pretty sure you can trigger a file download autmatically from JavaScript, but the user would get a file download dialog each time. It can't be hidden. The desktop app should be able to backup to a local file file automatically (or over SSH, though that seems redundant), but in a web browser the simplest option would probably be WebDAV.

If you're running in a web browser, your already have to trust whoever serves the client though, since they can just add something to it at any time.

[c33s]

first of all thank you at @t3chguy, i haven't seen such a good community manager and supporter in all the years. your answers come fast and are technically backedup. your endless patience and knowhow are awesome. big applause for you here and again thank you.

---

to clearify i am talking about the desktop app only. i don't use the webapp in the browser.

In the desktop app these limitations are not present but require much more additional work.

it would be very good to have at least a "space left on device"-check. killing the keys because of missing device space is really a terrible UX. i don't know how the key rotation work at all but a full harddisk should never lead to the deletion of existing keys, so at least all existing messages should be accessible/decryptable.

[t3chguy]

/me is not a Community Manager but a dev - but thanks :D

I think valid concerns and limitations are listed in this issue sufficiently for the Product team to plan a solution for.

[brumasribera]

Would there be a way to simply back up the conversations and get them when you log in into new devices. I am really not concerned about security and this huge complexity I find.

I only would like to be able to keep the whole history of messages all the time, and this seems to be impossible right now to be done on a human way, not having to export files with encrypted keys now and then.

[aaronraimist]

@brumasribera Yes. That is what the Secure Backup feature does. It automatically backs up your keys. As long as you provide your Security Key (or Security Phrase) each time you login, your keys will be automatically backed up and restored.

[brumasribera]

@aaronraimist Backing up key means also backing up messages, I am more concerned with messages and not so much on those keys.

For me, it would be just more simple to recover the messages normally once reconnected to your account and have this extra safety layer optional.

Or at least be able to disable it, as I find it a bit cumbersome for my normal use. I am of course only talking about UX experience for the user. Thank you for your time!

[aaronraimist]

@brumasribera

Backing up key means also backing up messages, I am more concerned with messages and not so much on those keys.

Well that's a different problem. If you don't trust your homeserver to not delete your messages then you probably want to be using the export chat feature. Go to the Room info page and then click Export chat. There is no feature to automatically do that though, that would be #1388.

For me, it would be just more simple to recover the messages normally once reconnected to your account and have this extra safety layer optional.

Encryption makes it very difficult to automatically restore messages. If you want messages to be automatically visible with no action other than logging in, your only real option today is to not encrypt the room. In the future matrix-org/matrix-spec-proposals#3265, matrix-org/matrix-spec-proposals#2957, or matrix-org/matrix-spec-proposals#3262 may solve this.