Isaac Gym Integration with wandb does not seem to record time or step correctly

[DanielTakeshi]

I am running PPO with wandb integration, but the statistics seem to not be recorded as intended.

Steps to reproduce: after installing following the IsaacGymEnvs instructions, run a command like this in the isaacgymenvs/ directory:

python train.py task=Ant headless=True wandb_activate=True wandb_entity=danieltakeshi wandb_project=isaac-gym

Where you can replace danieltakeshi with your username, and change isaac-gym to your project.

After I run this, the reward goes up (good) but I also see this on wandb:

The code is recording the reward as a function of iter, step, and time. It stores it in rl_games here:

https://github.com/Denys88/rl_games/blob/d8645b2678c0d8a6e98a6e3f2b17f0ecfbff71ad/rl_games/common/a2c_common.py#L947-L955

The code is storing the statistics with respect to different quantities (epoch, step, and time) to the self.writer which is a tensorboardX.SummaryWriter (link to docs). But the statistics on wandb seem to only show the x-axis as "iter" (which is the same as epoch_num here) and they don't show performance as a function of the step or time. Is there a way to address such an issue here? This also affects the episode_length statistic.

(Also posting on rl_games repo Denys88/rl_games#208)