[Bug]: spaces.Discrete - start argument ignored

[emrul]

🐛 Bug

I'm a bit new to SB3 and Gym (used it many years ago) so I raise this report hesitantly as it's quite possible I've done something wrong.

However, it seems to me that the start parameter of spaces.Action is ignored.

I've included a minimal repro below:

• Action space is 3
• start is -1
• It would be expected therefore for env to return values from [-1, 0, 1] and if I sample the gym directly this works
• What appears to be happening is env is returning values from [0, 1, 2]

Note: I'm using the branch of SB3 that has support for gymnasium - specifically I upgraded SB3 using pip install git+https://github.com/carlosluis/stable-baselines3@fix_tests. It isn't clear to me whether this is part of my issue or not.

To Reproduce

import sys
import numpy as np
import gymnasium as gym
sys.modules["gym"] = gym
from gymnasium import spaces
from stable_baselines3.common.vec_env import DummyVecEnv
from stable_baselines3 import PPO


class CustomGymEnv(gym.Env):
    def __init__(self, action_space, obs_space):
        """
        Custom gym environment for testing purposes
        """
        self.action_space = action_space
        self.observation_space = obs_space
        self.current_step = 0
        self.ep_length = 4

    def reset(self, seed=None):
        self.current_step = 0
        self._choose_next_state()
        return self.state

    def step(self, action):
        assert self.action_space.contains(action), f"{action} {type(action)} invalid"
        reward = float(np.random.rand())
        self._choose_next_state()
        self.current_step += 1
        done = self.current_step >= self.ep_length
        return self.state, reward, done, done, {}

    def _choose_next_state(self):
        self.state = self.observation_space.sample()

    def render(self, mode="human"):
        if mode == "rgb_array":
            return np.zeros((4, 4, 3))

    def seed(self, seed=None):
        if seed is not None:
            np.random.seed(seed)
            self.observation_space.seed(seed)


env_maker = lambda: CustomGymEnv(spaces.Discrete(3, start=-1), spaces.Box(low=np.zeros(2), high=np.ones(2)))
env = DummyVecEnv([env_maker])

model = PPO("MlpPolicy", env, verbose=1)
#model.learn(total_timesteps=1000)


obs = env.reset()
done = False
while not done:
    act, _states = model.predict(obs)
    obs, reward, done, info = env.step(act)
    env.render()

Relevant log output / Error message

File "....../test.py", line 26, in step 
assert self.action_space.contains(action), f"{action} {type(action)} invalid"
AssertionError: 2 <class 'numpy.int64'> invalid

### System Info

• OS: macOS-13.0.1-x86_64-i386-64bit Darwin Kernel Version 22.1.0: Sun Oct 9 20:14:54 PDT 2022; root:xnu-8792.41.9~2/RELEASE_X86_64
• Python: 3.10.8
• Stable-Baselines3: 2.0.0a0
• PyTorch: 1.13.1
• GPU Enabled: False
• Numpy: 1.24.1
• Gym: 0.26.2

### Checklist

- [X] I have checked that there is no similar [issue](https://github.com/DLR-RM/stable-baselines3/issues) in the repo
- [X] I have read the [documentation](https://stable-baselines3.readthedocs.io/en/master/)
- [X] I have provided a minimal working example to reproduce the bug
- [X] I've used the [markdown code blocks](https://help.github.com/en/articles/creating-and-highlighting-code-blocks) for both code and stack traces.

[qgallouedec]

Happy comeback then!

Discrete observation space with a non-zero start is not supported by Stable-Baselines3. You can use a wrapper or update your observation space.

It seems like you're using custom env. I highly recommend you to check your env before using.

from stable_baselines3.common.env_checker import check_env

env = CustomEnv()
check_env(env)

I'm pretty that it should raise the warning mentioned above ;)

[araffin]

Duplicate of #913
It seems that I added the warning for observation but forgot to do so for action space.

[emrul]

Thanks @araffin - I did look for a dup but couldn't find one. Anyway, agree that for me at least this is easy enough to work around.

[emrul]

@qgallouedec - thanks for pointing me to the env checker! That found a couple of problems that I've now addressed!!

[araffin]

It seems that I added the warning for observation but forgot to do so for action space.

Done =)