Wandb does not seem to record time or step correctly

[DanielTakeshi]

I am running PPO with wandb integration, but the statistics seem to not be recorded as intended.

I am testing this with Isaac Gym environments but I am unsure if this issue is specific to Isaac Gym.

Steps to reproduce: after installing following the IsaacGymEnvs instructions, run a command like this in the isaacgymenvs/ directory:

python train.py task=Ant headless=True wandb_activate=True wandb_entity=danieltakeshi wandb_project=isaac-gym

Where you can replace danieltakeshi with your username, and change isaac-gym to your project.

After I run this, the reward goes up (good) but I also see this on wandb:

The code is recording the reward as a function of iter, step, and time. It stores it in rl_games here:

rl_games/rl_games/common/a2c_common.py

Lines 947 to 955 in d8645b2

	for i in range(self.value_size):
	rewards_name = 'rewards' if i == 0 else 'rewards{0}'.format(i)
	self.writer.add_scalar(rewards_name + '/step'.format(i), mean_rewards[i], frame)
	self.writer.add_scalar(rewards_name + '/iter'.format(i), mean_rewards[i], epoch_num)
	self.writer.add_scalar(rewards_name + '/time'.format(i), mean_rewards[i], total_time)
	self.writer.add_scalar('episode_lengths/step', mean_lengths, frame)
	self.writer.add_scalar('episode_lengths/iter', mean_lengths, epoch_num)
	self.writer.add_scalar('episode_lengths/time', mean_lengths, total_time)

The code is storing the statistics with respect to different quantities (epoch, step, and time) to the self.writer which is a tensorboardX.SummaryWriter (link to docs). But the statistics on wandb seem to only show the x-axis as "iter" (which is the same as epoch_num here) and they don't show performance as a function of the step or time. Is there a way to address such an issue here?

(Also posting on the Isaac Gym repo isaac-sim/IsaacGymEnvs#87)

[Denys88]

@DanielTakeshi I am sorry I missed your issue.
@vwxyzjn could you take a look if you have free time?

[vwxyzjn]

try changing the x axis to global_step on the top right (there is a button)

[DanielTakeshi]

Sorry for my delayed repsonse as well, @Denys88 and @vwxyzjn.

It looks like we can adjust the x-values here:

So I think the intended usage here is that we are supposed to adjust rewards/time so that the x-axis has Wall Time and rewards/step so that it uses global_step? (Somewhat confusingly, rewards/iter seems fine with the normal Step though it is clear in the code that iter is supposed to refer to an epoch.)

It would be nice if there was a way to automatically set all three plots so that they use the appropriate x-axis at the start. I'm not sure if this function is available.

If this is the intended usage, feel free to close this issue report. Thanks!