FAQ

How to I modify an environment to make it easier to learn?

The short answer is: don't. Environments are intended to have various levels of difficulty, in order to benchmark the ability of reinforcement learning agents to solve them. Many of the environments are beyond the current state of the art, so don't expect to solve all of them. (If you do, please apply).

If you want to experiment with a variant of an environment that behaves differently, you should give it a new name so that you won't erroneously compare your agent running on an easy environment to someone else's agent running on a hard environment. For instance, the MountainCar environment is hard partly because there's a limit of 200 timesteps after which it resets to the beginning. Successful agents must solve it in less than 200 timesteps. For testing purposes, you could make a new environment MountainCarMyEasyVersion-v0 with different parameters by adapting one of the calls to register found in gym/gym/envs/init.py:

gym.envs.register(
    id='MountainCarMyEasyVersion-v0',
    entry_point='gym.envs.classic_control:MountainCarEnv',
    max_episode_steps=250,      # MountainCar-v0 uses 200
    reward_threshold=-110.0,
)
env = gym.make('MountainCarMyEasyVersion-v0')

Because these environment names are only known to your code, you won't be able to upload it to the scoreboard.