"ep_len_mean" and "ep_rew_mean" are inconsistent for Atari environments

[semihtasbas]

Hi,

I want to calculate ep_rew_mean on custom Atari environment. But while calculating this, I encounter a problem. I used following code.

env = make_atari_env('BreakoutNoFrameskip-v4', n_envs=16)
env = VecFrameStack(env, n_stack=4)
model = A2C("CnnPolicy", env, verbose=0)
model.learn(total_timesteps=int(5e6))

I debug this code and checked line 175 in "on_policy_algorithm.py":

new_obs, rewards, dones, infos = env.step(clipped_actions)

I count done condition with my hand, and it is around 30, while ep_len_mean is around 600. And there is also difference between ep_rew_mean and hand counted reward.
What is reason and how can I implement "ep_rew_mean" to my custom algorithm while using your Atari wrapper?

Thank you.

[araffin]

Duplicate of #181