Flag Done is not set

[hadarshavit]

Hi, we are using CULE with CUDA 11.3 and 2080 Ti GPU. We set the environment as follows:

AtariEnv('QbertNoFrameskip-v4', 64,
                     color_mode='gray', device=torch.device('cuda:0'), rescale=True,
                     frameskip=4, repeat_prob=0, episodic_life=True, max_noop_steps=30, max_episode_length=10000)

However the done flag is always set to false.
How can we solve it? Thanks!

[hadarshavit]

Minimal example:

import argparse
import numpy as np
from torchcule.atari import Env
parser = argparse.ArgumentParser(description="CuLE")
parser.add_argument("game", type=str, help="Atari game name (breakout)")
parser.add_argument("--n", type=int, default=20, help="Number of atari environments")
parser.add_argument("--s", type=int, default=200, help="Number steps/frames to generate per environment")
parser.add_argument("--c", type=str, default='rgb', help="Color mode (rgb or gray)")
parser.add_argument("--rescale", action='store_true', help="Resize output frames to 84x84 using bilinear interpolation")
args = parser.parse_args()
color_mode = args.c
num_envs = args.n
num_steps = args.s
env = Env(env_name=args.game, num_envs=num_envs, color_mode=color_mode, rescale=args.rescale, device='cuda:0')
observations = env.reset()
for _ in np.arange(num_steps):
    actions = env.sample_random_actions()
    observations, reward, done, info = env.step(actions)
    print(any(done))

Running it with python test.py Qbert --s 900000 gives just "False"
I tried also Pong and Freeway but got the same result
Tagging authors of the package @sdalton1 @ifrosio

[MatPoliquin]

@hadarshavit Just curious how much FPS you get with the Pong env on your 2080ti?