Prototype torch.distributed integration #158
Conversation
rl_games/common/a2c_common.py
Outdated
| @@ -966,7 +973,7 @@ def train(self): | |||
| update_time = 0 | |||
| if self.multi_gpu: | |||
| should_exit_t = torch.tensor(should_exit).float() | |||
| self.hvd.broadcast_value(should_exit_t, 'should_exit') | |||
| # self.hvd.broadcast_value(should_exit_t, 'should_exit') # what is the purpose of this? | |||
There was a problem hiding this comment.
here is a chance that one job will exit a little bit earlier, as result other jobs will crash.
There was a problem hiding this comment.
This method seems to be deprecated by horovod: the closest method I can find is https://horovod.readthedocs.io/en/stable/api.html#horovod.torch.broadcast_.
What exactly is this broadcast_value doing? If rank 0's should_exit_t=False and rank 1's should_exit_t=True, would broadcast_value overwrite rank 1's should_exit_t?
| @@ -0,0 +1,84 @@ | |||
| params: | |||
There was a problem hiding this comment.
btw could you use breakout from envpool?
There was a problem hiding this comment.
Envpool is actually more tricky. When prototyping with envpool it's actually slower with multi-GPU, at least out of the box. This is because envpool uses different threads and have complex interactions with these threads that are a bit difficult to control. For this reason, I have chosen the regular gym API for controlled performance.
|
cc @markelsanz14, I prototyped the
|
vwxyzjn
force-pushed
the
torch.distributed
branch
from
7a49df2
to
86f5e82
Compare
This PR Prototype
torch.distributedintegration for multi GPU