[Proposal] Support Mujoco 2.1

[krzentner]

Proposal

Update gym to support mujoco 2.1 (and a more recent mujoco_py version).

Motivation

Currently, gym officially only supports mujoco 1.5. It supported 2.0 in the past, but that was done without testing, and introduced a bug (#1541), which was closed by dropping mujoco 2.0 support in PR #1731.

In the intervening years, there has been a change floating around, which is believed to change mujoco_py to have the same behavior using mujoco >=2.0 and fix the bug. However, that change has not been merged into mujoco_py.

Up until now, labs have been addressing this (one line) flaw in mujoco_py by installing various forks, or by installing mujoco 1.5 explicitly. However, now that mujoco has been purchased by Deepmind, licenses for mujoco 1.5 are no longer available for purchase and are expiring. This leaves only installing mujoco_py forks, which is unfortunate.

Pitch

Fix mujoco_py and upgrade the gym dependency.
I have a PR open on the mujoco_py repo that fixes this problem.
If that is merged, I would be happy to test to see if it does fix the gym environments, and then open a PR to update the dependency in gym.

Alternatives

The alternative is to leave mujoco unsupported while alternative ways of running the environments are being implemented. However, this fix is a one line change with relatively low risk that is ready now, so it is probably worth pursuing before more long-term solutions.

[jkterry1]

All the mujoco environments are intended to be moved out of gym as legacy benchmarks in the near future, as discussed in these issues:

#2524
#2456

Additionally, OpenAI won't let me merge new things into the mujoco py repo or cut new releases

[jkterry1]

If at some point in the future I do get control of mujocopy environments, I'd be happy to merge PRs like yours to make using the environments easier for people though.