Documentation about security implications of functionality of E2E-encryption

[Gummikavalier]

Description:

Rocket.Chat needs more documentation about the implications and possible attack vectors for the implementation of the E2E-encryption feature to make sure users and maintainers of services would better understand them and plan their systems accordingly.

Steps to reproduce:

1. Go to https://docs.rocket.chat
2. Search for E2E or encryption
3. Check results

Expected behavior:

Depiction of how E2E-encryption works in Rocket.Chat

Actual behavior:

There is no documentation

Server Setup Information:

• Version of Rocket.Chat Server: 3.6.3
• Operating System: CentOS7
• Deployment Method: tar
• Number of Running Instances: 20
• DB Replicaset Oplog: yes
• NodeJS Version: 12.16.1
• MongoDB Version: 3.6

Client Setup Information

• Desktop App or Browser Version: Latest
• Operating System: Any

Additional context

Previous discussions about the E2E-implementation (stale):
#14829
#16153

This is a documentation request and suggestion about limitations of current E2E-encryption functionality to make sure that both maintainers and users of Rocket.Chat would understand how well they are protected when they are using this feature.

Please note that because there is no documentation currently available everything below is disputable until the developers confirm or deny these points themselves. It is known that both E2E-encryption private and public keys are saved into the database for easier deployment on the clients.

We did some testing and current implementation seems to work in a way that when one E2E-private key is completely reset and gets a new public key, following happens:

• At first all messages are invisible or in encrypted format on encrypted channel for the user who reset their key. User cannot send messages at this point on an encrypted channel.
• When any other existing members on the channel visits the channel once, their client end encrypts all messages on the channel again using the new public key of the member who reset the key.
• After the new public key has been added to the channel chain of keys on the channel, user with the new key can again read all messages on the channel, as well as send new messages on it.

Other notes regarding E2E-encryption in Rocket.Chat:

• Unlike in really strong E2E-encryption implementations, private keys are saved into the database on the server directly. This is probably for the ease of use for the users; doing it any other way would mean transferring keys between multiple clients manually by the users themselves.
• The password of the private key (aka E2E-password) is likely to be saved into the memory of client (browser or actual client), and not on the server to protect private keys from malicious administrators or hackers stealing the database and private keys along with it.
• It would be good to know whether the private key with its default E2E-password gets generated in the client end or at the server end before it gets saved into the database. Server to client direction would be somewhat unsafe, as both key and its password would traverse from the server to client end via "just" TLS-encrypted communication channels.
• Other server end weakness with generating keys would be with certain types of virtual servers where VMs lack possibility of generating more entropy to their random numbers from the signal noise of the physical ports of the underlying host server.
• Because the private keys are protected only by their passwords, they can be subjected to bruteforcing by administrators or hackers. Bruteforcing can crack any weak passwords in a matter of minutes.
• Because all messages on an encrypted channel are encrypted with all public keys of its members, the weakest link for the channel's security is always the weakest private key password of its all members.
• With private encrypted channels easiest way for malicious admin is to add their own public key to a private group channel is by changing the channel as public at first, then joining the channel, and then setting the channel as private again. Next time any of the original channel members visit the channel, they reroll the admin's public key into all existing messages automatically, again revealing their content for the admin.
• Above trick does not work with direct channels as direct channel memberships cannot be changed. At least not without tampering the database directly. Easiest method with direct channels is stealing the identity of the member and then replace their keys, although this leaves the subjected direct channel member with a non-working E2E-password until the key is reset again by them.

E2E-implementation protects the user in following situations:

1. Database has been stolen from the server. There could not be open clients with working private keys online at that point to decrypt the messages on the channels.
2. Messages in the transit between client and server get extra encryption layer on top of regular TLS encryption of the server and client communications.
3. Mobile notification system based on the Firebase cloud service as well as Rocket.Chat gateway service (hosted by the developers of Rocket.Chat and default gateway for non branded Rocket.Chat mobile app) through which the messages previously went in plain text (excluding TLS of the network connections).

E2E-implementation does not protect users in the following situations:

1. Malicious administrators. They can either reset the keys (and passwords) directly from the database, or just reset the password for the user, log in, reset the e2e-key in user's preferences. After that they'll only need to wait that any of the existing channel members sends a new message. At this point malicious admin's new public key based on its new private key and password will be taken into use on the channel, revealing all (new and old) encrypted messages on it.
2. Malicious administrators or hackers subject keys in the database to a brute-forcing attack, at which point all too weak keys would be found, compromising all channels the owners of these keys would be members of.

It is a compromise between usability and security:

As long as the automatic reset key functionality exists OR private keys are saved into the database OR old messages are rerolled automatically using new public keys, there cannot be protection against malicious administrators.

Also nothing protects against malicious administrators with web apps in general anyway: Since the web client code is downloaded from the server to the browser, it would be somewhat trivial to build such version of Rocket.Chat that would save the private key passwords, or even steal the private keys if weren't already in the database, instead of managing them just locally on the browser client.

To get perfect security would mean that private keys or their passwords would never be saved into the web browser, meaning users would have to use audited versions of such clients that don't download any code from any sources actively. Therefore it is reasonable that the attack vector of malicious administrators is ignored in the E2E-implementation. This just needs to be documented in a way that everyone understands the reasons and the implications these compromises pose for their security.

[Gummikavalier]

I just voice my opinion outside of the actual documentation request above as the implementation is mostly a design decision anyway.

From the pure security perspective I would not allow automatic re-rolling of old messages with any new public keys after these original messages have been sent.

Should the old messages be left encrypted only with their old keys, it would prevent of spying them very effectively. Members of the channel might be able to smell foul play in the event of fiddling with keys (their known passwords stop working), or when a long time inactive member suddenly becomes active on the channel.

Granted, people would lose the contents of old messages for good had they forgotten their E2E-key passwords, but that is the whole point of security. Also instead of just making people angry for losing their discussions in the unfortunate event of forgetting the password, it may give them some confidence that the method is working as they'd expect it to.

[frdmn]

Regarding missing docs about E2E, security team just released this article about E2E in our docs: https://docs.rocket.chat/guides/security/end-to-end-encryption-algorithms

[Gummikavalier]

Looks good and comprehensible to me. Exactly what was needed! Thanks! 👍

I'm closing this issue as it was purely about the documentation. Other issues can be opened specifically to discuss about possible attack vectors such as security of AES-CBC algorithm as well as automatic handling of session keys with newly introduced or reset member keys.

[geekgonecrazy]

I think most of this is answered in better detail in the doc. But felt like might be worth replying anyways.

Wasn’t involved in implementation or a security expert. Just someone that has been very curious in implementation and has dug a lot in code prior to the doc.

But from what I understand:

It would be good to know whether the private key with its default E2E-password gets generated in the client end or at the server end before it gets saved into the database. Server to client direction would be somewhat unsafe, as both key and its password would traverse from the server to client end via "just" TLS-encrypted communication channels.

Private and public key pair is generated client side. Server never sees unencrypted private key. Once generated the encrypted private key is saved on user object. So can pull up on mobile for instance where it would fetch and decrypt key locally.

The password of the private key (aka E2E-password) is likely to be saved into the memory of client (browser or actual client), and not on the server to protect private keys from malicious administrators or hackers stealing the database and private keys along with it.

The password is used to decrypt but is not stored.

Unlike in really strong E2E-encryption implementations, private keys are saved into the database on the server directly. This is probably for the ease of use for the users; doing it any other way would mean transferring keys between multiple clients manually by the users themselves.

Yes this is currently done to make a bit easier. This might be an area we are able to improve in the future. Maybe make it so you can choose to not have the private key synced.

Other server end weakness with generating keys would be with certain types of virtual servers where VMs lack possibility of generating more entropy to their random numbers from the signal noise of the physical ports of the underlying host server.

Since generated client side I think the entropy there should be decent. But I think on browser there are ways to increase this fairly easy. Like mouse position and other random factors.

Because the private keys are protected only by their passwords, they can be subjected to bruteforcing by administrators or hackers. Bruteforcing can crack any weak passwords in a matter of minutes.

I believe this is one of the reasons we generate an alpha numeric passcode that should be pretty difficult to brute force. Would be a place that would be good to get more feedback though.

---

Regarding rekeying messages. I think messages aren’t rekeyed. Because this would be extremely expensive on the rekeyers browser and would require them to pull all of the messages down and then re-encrypt them all. I might be wrong here

If I remember right there is essentially a room key for the group. The key is encrypted with each of the public keys of the members and stored on their subscription to the room.

So when you reset and an existing member visits room it takes users new public key and re encrypts that room key with their new public key.

I think a point that would help here is a way for the user to be prompted before this happens. Since it happens client side makes sense for them to be able to interject.

Also.. probably easy point to improve would be to send a message to let everyone know that user rekeyed.

[Gummikavalier]

Thanks for additional comments! Indeed my original assumption that all messages are encrypted using all public keys of all members on the channel was wrong. As @geekgonecrazy mentioned, this would have been very expensive on channels with lots of members. Instead E2E session key is used for that, and public keys are only used for distributing E2E session key safely.

I'm not a security expert in cryptography (just a sysadmin / solution designer). But from the document I now understand that E2E is working in this way:

1. It generates a private and public key for the users in their client ends safely.
2. Server is used to distribute the public keys for all channel members. The public keys are used to identify all members/clients properly (from encryption point of perspective).
3. Asymmetric encryption using public keys also allows clients to distribute actual E2E session key securely for all channel members/clients. E2E session key is also safely generated and encrypted in the client end.
4. When all room members have the E2E session key, it is used to encrypt and decrypt the messages. (At this point it is symmetric encryption.)

The end result is that the server is always acting merely as a transfer medium in KEX (key exchange) as well as for the encrypted messages. Just like in the encryption of network communications this part of the implementation is good and does not try to reinvent the wheel. It is a good solution. :)

---

The weak points are only those that allow circumventing or abusing the actual KEX part one way or other. Admins have this power currently by either impersonating users and resetting their keys, or by enforcing themselves as members on private channels.

Regarding handling of private key and above weak points:

Going into details I earlier mentioned I would not go into in this GitHub issue (sorry just thinking out loud here ;) ). Instead of saving the user's private key into the server database (although encrypted and thus still safely just like E2E session key is saved too), in the web client you could avoid it completely and offer a view in the web client that would allow users in hold of unencrypted private key a way of copying and pasting the private key to their desktop for safe keeping.

For mobile clients you could also show the private key in form of QR code, which then would be read and saved by the mobile client app. This way users would not have to worry about how to transfer the private key safely to the mobile device without using cloud services, USB-dongles or memory cards (latter two of which are not possible on all mobile devices).

But this would still not protect against malicious admins abusing their powers to impersonate other users on the server, and using this for smuggling in their own new public key based on their own private key.

---

For any users around there who do not bother thinking past the technical description in E2E documentation, it is important to understand that the original implications I mentioned in the first post of this GitHub issue still stand. While E2E implementation of Rocket.Chat is technically solid, in practice it is not completely that from the perspective of message security.

But also as mentioned in the first post, it does protect the content of messages against attacks during message transit, notifications system message transit and in case of database leaks.