[Bug] SB3 breaks with gym==0.20.0

[MaximilienLC]

🐛 Bug

Stable-baselines3==1.2.0 breaks with gym==0.20.0

To Reproduce

from stable_baselines3 import PPO

model = PPO.load("../../data/models/ppo/CartPole-v1")

AttributeError                            Traceback (most recent call last)
<ipython-input-1-124a176067c3> in <module>
      1 from stable_baselines3 import PPO
      2 
----> 3 model = PPO.load("../../data/models/ppo/CartPole-v1")

~/.local/lib/python3.6/site-packages/stable_baselines3/common/base_class.py in load(cls, path, env, device, custom_objects, **kwargs)
    686         model.__dict__.update(data)
    687         model.__dict__.update(kwargs)
--> 688         model._setup_model()
    689 
    690         # put state_dicts back in place

~/.local/lib/python3.6/site-packages/stable_baselines3/ppo/ppo.py in _setup_model(self)
    153 
    154     def _setup_model(self) -> None:
--> 155         super(PPO, self)._setup_model()
    156 
    157         # Initialize schedules for policy/value clipping

~/.local/lib/python3.6/site-packages/stable_baselines3/common/on_policy_algorithm.py in _setup_model(self)
    116             gamma=self.gamma,
    117             gae_lambda=self.gae_lambda,
--> 118             n_envs=self.n_envs,
    119         )
    120         self.policy = self.policy_class(  # pytype:disable=not-instantiable

~/.local/lib/python3.6/site-packages/stable_baselines3/common/buffers.py in __init__(self, buffer_size, observation_space, action_space, device, gae_lambda, gamma, n_envs)
    326     ):
    327 
--> 328         super(RolloutBuffer, self).__init__(buffer_size, observation_space, action_space, device, n_envs=n_envs)
    329         self.gae_lambda = gae_lambda
    330         self.gamma = gamma

~/.local/lib/python3.6/site-packages/stable_baselines3/common/buffers.py in __init__(self, buffer_size, observation_space, action_space, device, n_envs)
     47         self.observation_space = observation_space
     48         self.action_space = action_space
---> 49         self.obs_shape = get_obs_shape(observation_space)
     50 
     51         self.action_dim = get_action_dim(action_space)

~/.local/lib/python3.6/site-packages/stable_baselines3/common/preprocessing.py in get_obs_shape(observation_space)
    142     """
    143     if isinstance(observation_space, spaces.Box):
--> 144         return observation_space.shape
    145     elif isinstance(observation_space, spaces.Discrete):
    146         # Observation is an int

~/.local/lib/python3.6/site-packages/gym/spaces/space.py in shape(self)
     39     def shape(self):
     40         """Return the shape of the space as an immutable property"""
---> 41         return self._shape
     42 
     43     def sample(self):

AttributeError: 'Box' object has no attribute '_shape'

Expected behavior

I believe the gym/spaces/space.py file has been changed in gym==0.20.0.

The code runs as expected for gym==0.18.0.

Checklist

• I have checked that there is no similar issue in the repo (required)
• I have read the documentation (required)
• I have provided a minimal working example to reproduce the bug (required)

[araffin]

Hello,
thanks for creating the issue. unfortunately, it seems that the issue comes from gym, not SB3 (but I'll check how to fix things).

Related to cyprienc@e7a48d2 and #572

EDIT: in the meantime, I would recommend sticking to gym v0.19
the issue comes probably from the fact that the box object was pickled with gym 0.19 and unpickled with gym 0.20

[araffin]

I found a hack, that should probably go to gym repo:

from typing import Dict, Any
from gym.spaces import Space

def patch_setstate(self, state: Dict[str, Any]) -> None:
    """Restores pickled state."""
    if "shape" in state:
        state["_shape"] = state["shape"]
        del state["shape"]
    self.__dict__.update(state)

Space.__setstate__ = patch_setstate

[jkterry1]

@araffin can you please create a PR for this?

[araffin]

Fixed in openai/gym#2419