Games freeze on NVIDIA GeForce RTX 4050 Laptop if drive is in "On Demand" mode.

[se5a]

Proton Versions: 9.03, Experimental
Nvida Driver versions I've tried:
560.35.03unbunu0-gpu22.04.4
555.58.02unbunu0-gpu22.04.1

Games I've noticed this on:
Manor Lords
Zero Sievert
Satisfactory

Laptop is a
ASUSTeK product: ASUS TUF Gaming F15 FX507VU_TUF507VU v: 1.0
It has an intel gpu as well.

With Unreal games, (such as Manor lords) I have to use the "DXVK_FILTER_DEVICE_NAME="NVIDIA GeForce RTX 4050 Laptop GPU" launch option.

Games will play fine for a completely random amount of time, 10min, an hour or so, then freeze up and I'm forced to hard reboot with the power button.
This took me a long time to figure out what the problem was, I checked temperatures, memory, tried updating bios, tried a different graphics driver, tried a different proton version, nothing seemed to work, till I changed the nvidia profile.
When I switched the Nvidia profile from "On Demand" to "Performance" the problem appears to have gone away.
Games that were not using proton seemed to work fine, e.g. Factorio, Warthunder.