Multiple upgrades + web render_mode

[zorgluf]

Hi,
This is not really a pull request, more to show some changes I have made on my side and might be nice to share. Let me explain.
My initial goal was to introduce a web render gui for frouge game. I wished to use nicegui lib, but the python version on docker was too old. Tried to updgrade, but need also to upgrade stable diffusion, tensorflow, and so on with breaking changes every
where... So one thing to an other :

• upgraded python
• upgraded to stable baseline 3
• moving from tensorflow to pytorch
• using maskedPPO instead of custom masking in tensorflow that I didn't manage to reproduce with pytorch
• upgraded gym env
• added support to GPU, but removing MPI
• and finally enabling web rendering
  Quite a lot of breaking changes (only frouge works now)... not sure if you will be interested by part of this work. But if your are interested, I can work for more granular commit.
  Anyway, great great project you have, I had a lot of fun tuning models and playing games...