Element freezes or crashes when too many voice messages are in a chat

[AlexanderBabel]

Steps to reproduce

1. Create an encrypted chat with a few voice messages. I am not sure how many it takes but I guess 30+ is a good start. Each of them should have a length of more than one minute (I use a WhatsApp bridged chat)
2. Open the chat on Element Desktop
3. The app tries to fetch all voice messages and starts to lag (no or slow UI interaction) or crashes with a white screen
4. Reopen Element (and try to open another chat as fast as possible to prevent another crash)

Outcome

What did you expect?

The chat should be responsive and render only voice messages that are on-screen

What happened instead?

Slow chat loading and an app crash

Operating system

macOS (13.4 Macbook Air M2)

Application version

Element version: 1.11.34 Olm version: 3.2.14

How did you install the app?

Element Website

Homeserver

Synapse 1.86.0

Will you send logs?

Yes

[weeman1337]

I can reproduce this. Just upload audio files and record voice messages in a room. Memory usage will grow up fast to several gigabytes. Then it crashes.

[weeman1337]

O-Occassional because some people tend to send many voice messages and the may see it regularly. S-Critical because you cannot use those rooms any more. I am currently trying to get out of my test room to be able to use Element again 😐

[sosnik]

Can reproduce on web and desktop. iOS Client not affected.

[t3chguy]

Related #19756

[germain-gg]

The white screen is a symptom of Electron, but this would equally happen when running in the browser.

Here's a few thoughts on my initial investifation of the problem,
Voice messages waveforms are generated client-side. When receiving multiple messages at the same time, this intensive process can lead to the page crashing unfortunately.

An avenue to fix this would be to move the waveform generation to a web worker and potentially add some sort of "max waveform generation" in parralel system.

Contributions are of course welcome to fix this issue

[t3chguy]

An avenue to fix this would be to move the waveform generation to a web worker

This already is the case https://github.com/matrix-org/matrix-react-sdk/blob/develop/src/workers/playback.worker.ts

[asodyora]

I don't know if it's helpful but I'd like to add:
This doesn't just happen with many voice messages in one chat, it also happens when it's just 1 very long voice message (think 15 minutes).

[weeman1337]

I can still reproduce it. Element runs into a memory limit and then crashes. Interestingly this also happens for audio files without a waveform:

[weeman1337]

The problem is that we are currently loading all files into memory: https://github.com/matrix-org/matrix-react-sdk/blob/491b43ef095f6a174755e9e8977218bfe52b98fa/src/components/views/messages/MAudioBody.tsx#L54

sourceBlob is already a „lazy value“. But with the code linked above it will become non-lazy.

I would say a solution can be to extend Playback to work with lazy values as well and then pass down the lazy values. A difference then would be the decryption error handling. Playback needs to provide error feedback and the tile should display it.

[ivy-reps]

The problem is that we are currently loading all files into memory: https://github.com/matrix-org/matrix-react-sdk/blob/491b43ef095f6a174755e9e8977218bfe52b98fa/src/components/views/messages/MAudioBody.tsx#L54

sourceBlob is already a „lazy value“. But with the code linked above it will become non-lazy.

I would say a solution can be to extend Playback to work with lazy values as well and then pass down the lazy values. A difference then would be the decryption error handling. Playback needs to provide error feedback and the tile should display it.

seems so, but I'm not sure that this was resolved, and to me that issue is critical because I'm not bale to use Element on Desktop. Immediately it is takes all RAM and I'm seeing white background in Element Desktop Window. And I'm forced to remove all voice messages to make it to work.

[MartenKahr]

Still seems to be happening: I can produce a consistent OOM hang of element-desktop by scrolling up to a part of chat history where someone quoted a series of uploaded audio files in quick succession. Memory usage spikes to 6.1GB (out of my computer's 8GB of RAM, and compared to a "normal" of around 100-300MB) and the chat view starts lagging as I do anything (scrolling, swapping rooms etc.) a couple seconds after seeing the quoted audio files. After about 5 to 10 seconds of the stuttering state, Element starts showing only a blank white screen and becomes unresponsive. Element did drop the excessive RAM usage in the unresponsive state after about a minute, but remained unresponsive for about five minutes before I decided it probably wasn't going to recover and killed it.

[sosnik]

Believe me, physical RAM is not the issue. It still crashes even with 128GB and I 100% don't have 128GB of voice messages.

…

On Sun, Apr 7, 2024 at 7:05 PM Marten Kähr ***@***.***> wrote: Still seems to be happening: I can produce a consistent OOM crash of element-desktop by scrolling up to a part of chat history where someone quoted a series of uploaded audio files in quick succession. Memory usage spikes to 6.1GB (out of my computer's 8GB of RAM, and compared to a "normal" of around 100-300MB) and the chat view starts lagging as I do anything (scrolling, swapping rooms etc.) a couple seconds after seeing the quoted audio files. After about 5 to 10 seconds of the stuttering state, Element starts showing only a blank white screen and becomes unresponsive. Element did drop the excessive RAM usage in the unresponsive state after about a minute, but remained unresponsive for about five minutes before I decided it probably wasn't going to recover and killed it. — Reply to this email directly, view it on GitHub <#26239 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGMXXMUWRQ65NNJSU3C3DKLY4ED65AVCNFSM6AAAAAA5HRX3I6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGM3TQMJUGE> . You are receiving this because you commented.Message ID: ***@***.***>

[MartenKahr]

Oh, definitely not. There's no way five or six sub-10-minute mp3 files are 6GB either. Something about audio uploads in quick succession (whether they're voice messages or something else) causes Element to fill all memory it can grab with... something it fails to use or release, and crash when the system inevitably stops giving it more. That said, I've done a bit more testing. I've let it stay on the unresponsive whitescreen for a good 30 minutes, and it's safe to say element doesn't recover gracefully from whatever happens here, even if it does release the crazy peak memory allocation pretty quickly after hanging. I've also managed to capture this during the peak of memory usage:

Whatever it's doing, it's clearly trying to allocate a downright unreasonable amount of RAM for whatever it's trying (and failing) to do when it encounters that part of room history with the quoted audio uploads. If anyone has suggestions for any useful testing/logging I'm not an advanced enough of a user to think of on my own, let me know and I'll try and do it, since I can reproduce the issue on demand just by scrolling up in a particular room.

[YourSandwich]

Same issue for me, 2 years of voice message history has about 3gb in size. I have exactly the same behavior as described here, but it works in the WebBrowser version of Element. Other Matrix Clients are unaffected.

[ivy-reps]

Here we go!

[ivy-reps]

YES THAT WORKS! YEEEEEEEEHHHHHHHHHHAAAAAAAA!!!!!!!!!!!!!!!!!!!!!!!!

[qupig]

It looks like 1.11.67 does fix the issue.
But it's worth noting that there's still quite a bit of memory usage, up to around 2GB(even 3.5GB) instead of the previous 5GB.

[MartenKahr]

Desktop client still stutters and tries to hog a whole lot of memory (see behavior in the memory maps picture from my Apr 7 post; still happening, but stops at 32GiB) when a lot of audio files in quick succession are uploaded, but the client doesn't soft-crash into a broken state anymore.

[t3chguy]

This is still a problem