Upgrade gym version to 0.26.1

[AdityaGudimella]

Problem Description

Upgrade gym version used in cleanrl from 0.23.1 to 0.25.1

Checklist

• I have installed dependencies via poetry install (see CleanRL's installation guideline.
• I have checked that there is no similar issue in the repo (required)

Possible Solution

StabeBaselines' ReplayBuffer currently does not support the new format returned by gym.Env.step. Their step api changed from:
obs, rew, done, info = env.step(action) to obs, rew, terminated, truncated, info = env.step(action).
We would need to implement a slightly modified version of the ReplayBuffer in cleanRL itself. Other than this, the changes required are minimal.

I can submit an initial PR with changes required for SAC if you're interested.

[vwxyzjn]

Update on the ticket - the current gym master is set to release 0.26.0 which enables obs, rew, terminated, truncated, info = env.step(action) by default.

[vwxyzjn]

#424 closes this issue to use gymnasium.